-- Logs begin at Tue 2022-09-13 13:32:06 UTC, end at Tue 2022-09-13 14:53:17 UTC. -- Sep 13 13:57:49.986439 dv-cinder-857434-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 13:57:49.997340 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 13:57:49 2022] *** Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: nodename: dv-cinder-857434-1-iscsi Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: machine: x86_64 Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: clock source: unix Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: pcre jit disabled Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: detected number of CPU cores: 4 Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: current working directory: / Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: detected binary path: /usr/bin/uwsgi-core Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: your processes number limit is 47680 Sep 13 13:57:49.999595 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: your memory page size is 4096 bytes Sep 13 13:57:50.000265 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: detected max file descriptor number: 2048 Sep 13 13:57:50.000265 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: lock engine: pthread robust mutexes Sep 13 13:57:50.000265 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: thunder lock: enabled Sep 13 13:57:50.000265 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 13:57:50.000265 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 13:57:50.038727 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: Python main interpreter initialized at 0x56155f3979c0 Sep 13 13:57:50.038727 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: python threads support enabled Sep 13 13:57:50.038727 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: your server socket listen backlog is limited to 100 connections Sep 13 13:57:50.038727 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: your mercy for graceful operations on workers is 90 seconds Sep 13 13:57:50.039081 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 13:57:50.039162 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: *** Operational MODE: preforking *** Sep 13 13:57:50.039395 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: *** uWSGI is running in multiple interpreter mode *** Sep 13 13:57:50.039395 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: spawned uWSGI master process (pid: 81593) Sep 13 13:57:50.039597 dv-cinder-857434-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 13:57:50.040689 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: spawned uWSGI worker 1 (pid: 81605, cores: 1) Sep 13 13:57:50.040689 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: spawned uWSGI worker 2 (pid: 81606, cores: 1) Sep 13 13:57:50.040689 dv-cinder-857434-1-iscsi devstack@keystone.service[81593]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 13:57:54.334723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 13:57:54.334723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Sep 13 13:57:54.335529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 13:57:54.335529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Sep 13 13:57:54.335673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 13:57:54.335673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: from cryptography.utils import int_from_bytes Sep 13 13:57:54.336092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 13:57:54.336092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: from cryptography.utils import int_from_bytes Sep 13 13:57:54.385939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 13:57:54.386133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 13:57:54.386312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] command line args: [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 13:57:54.386452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 13:57:54.386576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 13:57:54.386660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 13:57:54.386740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ================================================================================ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 13:57:54.386826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] command line args: [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 13:57:54.386988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 13:57:54.387226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ================================================================================ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 13:57:54.387314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_dir = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] admin_token = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_source = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_dir = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] debug = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.387990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] config_source = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] debug = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.388959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.389095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.389172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.389335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_config_append = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.389479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.389602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.389718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_dir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.389718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_config_append = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_dir = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.390978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.391955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392239 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.392960 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.393105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.393197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.393339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.393572 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.393648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.393849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.393954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] publish_errors = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394741 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.394973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] publish_errors = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395212 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] standard_threads = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.395996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_journal = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_json = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_stderr = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] standard_threads = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396540 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_syslog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.396931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.397044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.397124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_journal = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.397240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.397318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_json = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.397418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.397500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_stderr = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.397572 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] use_syslog = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.397641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.397833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 13:57:54.397925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.398912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.399075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.399402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.399575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.399682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.399825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.399913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.400995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.401967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.402959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.403981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.404109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.404188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.404356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.404442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.404587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.404664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.405325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.405419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.405591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.405683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.405888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.406888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407726 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.407957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.408973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.409920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.410858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.411927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.412962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.413888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.414858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.415947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.416965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.417914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.418854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.419932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.420011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.420158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.420233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.420378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.420478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.421190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.421364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.421453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.421669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.421782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.421942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422731 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.422964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.423913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.external = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.password = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.424874 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.token = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.external = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.425984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426230 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426312 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.password = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] auth.token = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.426928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.427965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.428914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.429927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.430904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.431856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.432977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433529 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.433952 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.434949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.435901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.436048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.436148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.436334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.436458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.436591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.436841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.436944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437576 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.437919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.438858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.439942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.440970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441675 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.441936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.442857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.443929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.444887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445180 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.445948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.446992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.447922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.448843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.449975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.450978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.451131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.451218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.451363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.451442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.451604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.451695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.451886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.452927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.453995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.454943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.455904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.456986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.457948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.458986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.459930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.460905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.461977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.462964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.463923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464648 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.464900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.465988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.466940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.467028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.467173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.467252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.467943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.468967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469869 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.469960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470191 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.470945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.471899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.472974 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.473939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474863 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.474942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.475023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 13:57:54.475098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.475326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.475563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.475808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.476040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.476280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.476516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 13:57:54.476684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81606) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 13:57:54.504646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.504871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.505089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.505736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.505984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.506211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.513745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.514589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.543055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.543281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.543420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.543654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.597448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.597777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.647037 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.647242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.647399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.647604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.805464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.805696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.805938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.806806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.807022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.830277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.830387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.830508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.830611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.830732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.830816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.830953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.831036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.831170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.831254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.831397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.831476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.831603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.831689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.843512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.843608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.856935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.857025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.857148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.857230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.857362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.857444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81606) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.857578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 13:57:54.874150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.874360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 13:57:54.874515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.874701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.874784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 13:57:54.874901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 13:57:54.875046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.875179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.875262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 13:57:54.875446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.875536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.875709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.875791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.875971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.876053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.876240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.885635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.885836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.885946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.886117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.886224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.886388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.886510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.886670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.894095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.894450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.897002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.897290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.897372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.897579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.897662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.897891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.897988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.898177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.898260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.898442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.898524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.898717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.898801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.898995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.899077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.899551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.899646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.899810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.899898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.900076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.900157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.900340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.923832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.923994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.925308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.925450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.925595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.925754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.930323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.930407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.930611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.930693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.930898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.930980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.937340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.937621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.937711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.937970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.943245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.943543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.943633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.943835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.943937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.944121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.944213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.944395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.944486 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.944667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.944767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.944938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.945030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.945208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.945298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.945505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.961240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.961519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.961842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.962128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.965399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.966025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.968705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.969000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.969394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.969694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.973220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.973962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.976364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.977338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.979593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.979895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.980178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.980462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.980577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.980741 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.980880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.981043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.981170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.981460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.981763 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.982058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.997351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.997639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.997932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.998197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.998670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:54.999477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:54.999751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.000018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.006156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.007070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.007723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.008633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.009243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.009519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.010205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.010487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.012199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.012482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.012764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.013056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.013138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.013425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.013509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.013719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.014036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.014589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.025314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.025597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.025907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.026191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.026272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.026487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.026570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.026771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.026854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.027063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.027155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.027447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.027741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.028023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.042487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.043592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.045391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.047109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.048909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.050159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.050427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.051609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.051889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.052172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.052566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.053091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.053378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.053799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.058924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.060187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.060373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.062363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.063920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.064316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.064603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.064879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.065255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.065657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.065973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.066252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.072761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.073054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.073336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.074094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.074400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.074687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.080452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.080736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.081017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.081291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.081576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.081787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81606) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 13:57:55.081873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.082077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.082162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.082369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.082452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.082649 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.082739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.082920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.083011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.083195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.083471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.083556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.083750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.083840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.084032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.084122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.084342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.084424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.084647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.084914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.085184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.085457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81606) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 13:57:55.119034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56155f3979c0 pid: 81605 (default app) Sep 13 13:57:55.119982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56155f3979c0 pid: 81606 (default app) Sep 13 13:57:55.122354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6fa4-dcb9-42e9-b6e6-071647cf2bde None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:57:55.122642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6fa4-dcb9-42e9-b6e6-071647cf2bde None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:57:55.122914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af6fa4-dcb9-42e9-b6e6-071647cf2bde None None] PATH_INFO: `/v3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:57:55.127640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Tue Sep 13 13:57:55 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:01.877525 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e02c95-016c-4e0e-9610-5ae98955fd90 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:01.878057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e02c95-016c-4e0e-9610-5ae98955fd90 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:01.878057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e02c95-016c-4e0e-9610-5ae98955fd90 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:01.882433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:01.887672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a910238-d567-434d-aca9-08b96538d277 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:01.888103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a910238-d567-434d-aca9-08b96538d277 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:01.888248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a910238-d567-434d-aca9-08b96538d277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:01.958905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: INFO dbcounter [None req-4a910238-d567-434d-aca9-08b96538d277 None None] Registered counter for database keystone Sep 13 13:58:01.977303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG oslo_db.sqlalchemy.engines [None req-4a910238-d567-434d-aca9-08b96538d277 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 13:58:01.983608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writer thread running {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 13:58:02.170406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-4a910238-d567-434d-aca9-08b96538d277 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81605) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 13:58:02.170864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-4a910238-d567-434d-aca9-08b96538d277 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81605) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 13:58:02.228600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4a910238-d567-434d-aca9-08b96538d277 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:02.230238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a910238-d567-434d-aca9-08b96538d277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:02.360105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 13:58:02.666561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e02c95-016c-4e0e-9610-5ae98955fd90 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:02.737370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: INFO dbcounter [None req-c0e02c95-016c-4e0e-9610-5ae98955fd90 None None] Registered counter for database keystone Sep 13 13:58:02.759086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG oslo_db.sqlalchemy.engines [None req-c0e02c95-016c-4e0e-9610-5ae98955fd90 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81606) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 13:58:02.766051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writer thread running {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 13:58:02.888227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e02c95-016c-4e0e-9610-5ae98955fd90 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:02.892791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b39c86-8a37-42ad-befc-326374d16d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:02.893419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b39c86-8a37-42ad-befc-326374d16d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:02.893702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b39c86-8a37-42ad-befc-326374d16d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:02.894015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b39c86-8a37-42ad-befc-326374d16d48 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:02.902490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b39c86-8a37-42ad-befc-326374d16d48 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:02.955672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING keystone.common.rbac_enforcer.enforcer [None req-33b39c86-8a37-42ad-befc-326374d16d48 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 13:58:02.956029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b39c86-8a37-42ad-befc-326374d16d48 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:02.965346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Sep 13 13:58:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:58:02.972117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a910238-d567-434d-aca9-08b96538d277 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:02.989234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a910238-d567-434d-aca9-08b96538d277 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:02.993604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a62bba-2d2b-40ce-9331-7d423bf096ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:02.994734 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62bba-2d2b-40ce-9331-7d423bf096ff admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:02.995024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62bba-2d2b-40ce-9331-7d423bf096ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:02.995304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a62bba-2d2b-40ce-9331-7d423bf096ff admin admin] PATH_INFO: `/v3/projects/4d23d643906b4c84bfbe35da532a7c31` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:02.996867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a62bba-2d2b-40ce-9331-7d423bf096ff admin admin] RBAC: Authorizing `identity:get_project(project_id=4d23d643906b4c84bfbe35da532a7c31)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:03.050481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING keystone.common.rbac_enforcer.enforcer [None req-86a62bba-2d2b-40ce-9331-7d423bf096ff admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 13:58:03.050822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a62bba-2d2b-40ce-9331-7d423bf096ff admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:03.055322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Tue Sep 13 13:58:02 2022] GET /identity/v3/projects/4d23d643906b4c84bfbe35da532a7c31 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:05.161647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade277-da0b-4ada-8df0-31fa1e59f474 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:05.162185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade277-da0b-4ada-8df0-31fa1e59f474 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:05.162267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ade277-da0b-4ada-8df0-31fa1e59f474 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:05.166304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:05.172601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c2b6-b598-488d-9de0-7693f4d22967 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:05.172999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c2b6-b598-488d-9de0-7693f4d22967 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:05.173146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c2b6-b598-488d-9de0-7693f4d22967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:05.192902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6549c2b6-b598-488d-9de0-7693f4d22967 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:05.194447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6549c2b6-b598-488d-9de0-7693f4d22967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:05.202448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 13:58:05.472506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ade277-da0b-4ada-8df0-31fa1e59f474 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:05.484608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ade277-da0b-4ada-8df0-31fa1e59f474 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:05.488788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32df384-eaba-466c-8801-a2895ab2e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:05.489378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df384-eaba-466c-8801-a2895ab2e861 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:05.489654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df384-eaba-466c-8801-a2895ab2e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:05.489955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df384-eaba-466c-8801-a2895ab2e861 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:05.497427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32df384-eaba-466c-8801-a2895ab2e861 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:05.499236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32df384-eaba-466c-8801-a2895ab2e861 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:05.507761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Sep 13 13:58:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 13:58:05.513067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549c2b6-b598-488d-9de0-7693f4d22967 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:05.522791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549c2b6-b598-488d-9de0-7693f4d22967 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:05.527589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02f418-53a4-439a-8cd6-f31c20499ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:05.528220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f418-53a4-439a-8cd6-f31c20499ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:05.528504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f418-53a4-439a-8cd6-f31c20499ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:05.528779 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f418-53a4-439a-8cd6-f31c20499ab8 admin admin] PATH_INFO: `/v3/users/9242a7b2cdf946a38262cb2b38f5cb44` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:05.530790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02f418-53a4-439a-8cd6-f31c20499ab8 admin admin] RBAC: Authorizing `identity:get_user(user_id=9242a7b2cdf946a38262cb2b38f5cb44)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:05.532723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02f418-53a4-439a-8cd6-f31c20499ab8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:05.534291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Tue Sep 13 13:58:05 2022] GET /identity/v3/users/9242a7b2cdf946a38262cb2b38f5cb44 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:07.737882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b889d-eaa1-4d5c-9198-41860e45f240 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:07.738446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b889d-eaa1-4d5c-9198-41860e45f240 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:07.738446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b889d-eaa1-4d5c-9198-41860e45f240 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:07.739759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:07.744921 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7201-bf4c-4075-960f-83da6a8dcbbc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:07.745333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7201-bf4c-4075-960f-83da6a8dcbbc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:07.745591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a7201-bf4c-4075-960f-83da6a8dcbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:07.764269 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4b6a7201-bf4c-4075-960f-83da6a8dcbbc None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:07.765691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b6a7201-bf4c-4075-960f-83da6a8dcbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:07.793533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a05c9-230d-47e1-9232-2db2271d2f02 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:07.793770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a05c9-230d-47e1-9232-2db2271d2f02 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:07.794050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a05c9-230d-47e1-9232-2db2271d2f02 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:07.795363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:07.801307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:07.801590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:07.801903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:07.844543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG passlib.handlers.bcrypt [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81606) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 13:58:07.844925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG passlib.handlers.bcrypt [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81606) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 13:58:07.848510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:07.894697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:07.896288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:07.995381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:08.116066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7201-bf4c-4075-960f-83da6a8dcbbc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.140137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:08.221266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a7201-bf4c-4075-960f-83da6a8dcbbc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.254890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.255520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.255801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.256076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.267745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:08.271736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:08.276932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:08.278102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:08.279355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:08.284405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.365137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:08.371627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62a287-6aad-405a-8234-d99cb7ffa565 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.373879 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:08.383775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcbe34-a64f-46a4-93ac-319995fa7fdd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.411533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.412389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.412673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.412952 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.418666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.419272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.419557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.419835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.420512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:08.420895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:08.422508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:08.434350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:08.440263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.480172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:08.481704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:08.543259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f7f15-1d0d-45bc-93b9-adfb0e31fb82 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.576808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.578105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.578390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.578667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] PATH_INFO: `/v3/users/9242a7b2cdf946a38262cb2b38f5cb44` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.579054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 13:58:08.579960 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] RBAC: Authorizing `identity:get_user(user_id=9242a7b2cdf946a38262cb2b38f5cb44)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:08.581427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:08.583041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/users/9242a7b2cdf946a38262cb2b38f5cb44 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:08.584587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.587382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:08.589010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.603908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:08.684680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2600ed2-ed47-4cc8-9b6c-dd90dada514e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.688096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2aa3e-3525-4580-a5c7-12e7e049aae4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.718918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.719771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.720051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.720322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.722819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.723433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.723712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.723983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.725079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:08.726552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:08.728063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:08.733943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.738608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:08.740067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:58:08.759013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:08.760114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:58:08.761129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:58:08.761129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:58:08.761129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:08.761129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:08.766455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:08.820418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:08.823230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:08.840620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf943b95-9719-49b7-8c91-450d248ca557 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.871278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0796f63-465b-4117-b0a4-8a759420f70a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:08.874676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39776731-b08d-4e08-a71d-ea2525d10695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.875284 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39776731-b08d-4e08-a71d-ea2525d10695 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.875564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39776731-b08d-4e08-a71d-ea2525d10695 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.875838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39776731-b08d-4e08-a71d-ea2525d10695 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.876561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39776731-b08d-4e08-a71d-ea2525d10695 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.domain.id=default, user.id=9242a7b2cdf946a38262cb2b38f5cb44) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:08.876868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39776731-b08d-4e08-a71d-ea2525d10695 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:08.878326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39776731-b08d-4e08-a71d-ea2525d10695 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:08.884873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/role_assignments?user.id=9242a7b2cdf946a38262cb2b38f5cb44&scope.domain.id=default&role.id=96af379f1477489b8542698d3c012f1a => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:08.906971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a171a2-da56-468b-8137-f23abb534bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:08.907606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a171a2-da56-468b-8137-f23abb534bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:08.907890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a171a2-da56-468b-8137-f23abb534bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:08.908168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a171a2-da56-468b-8137-f23abb534bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:08.908838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a171a2-da56-468b-8137-f23abb534bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:08.909147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a171a2-da56-468b-8137-f23abb534bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:08.910641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a171a2-da56-468b-8137-f23abb534bb4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:08.930125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:11.382157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70e7a5-741a-4bba-bc6c-35ea7e214722 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:11.382902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70e7a5-741a-4bba-bc6c-35ea7e214722 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:11.382902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70e7a5-741a-4bba-bc6c-35ea7e214722 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:11.384336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:11.390295 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b6d1-2125-431b-a54b-68d8f06b2402 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:11.390764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b6d1-2125-431b-a54b-68d8f06b2402 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:11.390859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2b6d1-2125-431b-a54b-68d8f06b2402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:11.414505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a1a2b6d1-2125-431b-a54b-68d8f06b2402 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:11.415782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a1a2b6d1-2125-431b-a54b-68d8f06b2402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:11.507725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 11/23] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:11.829399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97586ec-ddf0-4175-af5e-849ae83f6933 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:11.829789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97586ec-ddf0-4175-af5e-849ae83f6933 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:11.830053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97586ec-ddf0-4175-af5e-849ae83f6933 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:11.832411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 13/24] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:11.838445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b047b-19c5-4e7c-aac4-a7a5efb60697 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:11.838924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b047b-19c5-4e7c-aac4-a7a5efb60697 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:11.839467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b047b-19c5-4e7c-aac4-a7a5efb60697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:11.865363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6a3b047b-19c5-4e7c-aac4-a7a5efb60697 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:11.866859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6a3b047b-19c5-4e7c-aac4-a7a5efb60697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:11.941535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:11.976005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97586ec-ddf0-4175-af5e-849ae83f6933 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:12.005753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 12/25] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:12.045206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d887037-91d6-43ec-9608-15eaee2fe1ee None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.045489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d887037-91d6-43ec-9608-15eaee2fe1ee None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.046642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d887037-91d6-43ec-9608-15eaee2fe1ee None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.048427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 13/26] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:12.057187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925994d-d845-45af-a072-5c7499ff2d2c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.057836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925994d-d845-45af-a072-5c7499ff2d2c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.058340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925994d-d845-45af-a072-5c7499ff2d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.104915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4925994d-d845-45af-a072-5c7499ff2d2c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:12.107101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4925994d-d845-45af-a072-5c7499ff2d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:12.183954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97586ec-ddf0-4175-af5e-849ae83f6933 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:12.244297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 14/27] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 13:58:12.245596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5f639-e655-4c05-9017-9be483ba6a7e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.245911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5f639-e655-4c05-9017-9be483ba6a7e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.246183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5f639-e655-4c05-9017-9be483ba6a7e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.247619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 15/28] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:12.255960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4080687-bff9-4039-97b7-8c6e25bb9b3f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.256454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4080687-bff9-4039-97b7-8c6e25bb9b3f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.256889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4080687-bff9-4039-97b7-8c6e25bb9b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.258381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:12.259018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.259300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.259569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.279605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:12.289755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:12.290831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:12.291805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 14/29] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:12.293075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:12.300887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:12.306779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e4080687-bff9-4039-97b7-8c6e25bb9b3f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:12.308290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e4080687-bff9-4039-97b7-8c6e25bb9b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:12.426657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 16/30] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 13:58:12.428143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4080687-bff9-4039-97b7-8c6e25bb9b3f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:12.488502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c0b16-31cd-4b5b-97a1-ceb83b8619dc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:12.516614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:12.568227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:12.569071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.569510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.569955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.570788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:12.571228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:12.572757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:12.586006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 15/31] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:12.591905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:12.665051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4080687-bff9-4039-97b7-8c6e25bb9b3f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:12.740096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:12.748013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:12.748642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.749472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.749847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.750685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:12.752108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:12.792719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63502c-7c97-4979-aecf-9052a6b30442 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:12.840544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:12.841334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:12.841903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:12.842373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] PATH_INFO: `/v3/users/9242a7b2cdf946a38262cb2b38f5cb44` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:12.843894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] RBAC: Authorizing `identity:get_user(user_id=9242a7b2cdf946a38262cb2b38f5cb44)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:12.845607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:12.847493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 16/32] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:12 2022] GET /identity/v3/users/9242a7b2cdf946a38262cb2b38f5cb44 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:12.848464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:12.907010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 17/33] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 13 13:58:12 2022] POST /identity/v3/roles => generated 219 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:12.908345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:12.990914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d734d-98a5-413a-92a4-5a962af41423 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:13.049486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:13.050143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.050427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.050697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.051379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:13.052989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:13.058096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:13.116692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:13.127817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b642f4-4fb4-408f-85f1-6d933e25e65e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:13.169367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:13.170288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.170805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.171730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.182801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:13.184584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:58:13.207770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:13.208848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:58:13.209892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:58:13.209892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:58:13.209892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:13.211485 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 18/34] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:13.212639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:13.231441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 17/35] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 13 13:58:12 2022] POST /identity/v3/roles => generated 225 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:13.232426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:13.379183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a7c1a-3934-4577-b456-3f4809efb2d5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:13.429091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e943550-47c4-4f00-8883-676a0a2ad4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:13.429891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e943550-47c4-4f00-8883-676a0a2ad4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.430083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e943550-47c4-4f00-8883-676a0a2ad4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.430330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e943550-47c4-4f00-8883-676a0a2ad4a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.431218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c207a-86dc-4c7d-89a9-b83db3ef55d8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:13.431792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e943550-47c4-4f00-8883-676a0a2ad4a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:13.433607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e943550-47c4-4f00-8883-676a0a2ad4a4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:13.441134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 19/36] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:13.441134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ecf51-3b87-46f2-8a3f-819592b31e0a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.441134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ecf51-3b87-46f2-8a3f-819592b31e0a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.441134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ecf51-3b87-46f2-8a3f-819592b31e0a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.441134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 20/37] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:13.449558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ecf51-3b87-46f2-8a3f-819592b31e0a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:13.474001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:13.484763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09580090-39ca-4f49-9da9-0a56fb2d6b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:13.486029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09580090-39ca-4f49-9da9-0a56fb2d6b7c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.486461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09580090-39ca-4f49-9da9-0a56fb2d6b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.487109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09580090-39ca-4f49-9da9-0a56fb2d6b7c None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.488127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09580090-39ca-4f49-9da9-0a56fb2d6b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:13.488880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09580090-39ca-4f49-9da9-0a56fb2d6b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:13.491201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09580090-39ca-4f49-9da9-0a56fb2d6b7c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:13.511132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 18/38] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:13.512550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623f25-e431-4c0c-b221-94805f26ef54 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.512844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623f25-e431-4c0c-b221-94805f26ef54 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.513122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623f25-e431-4c0c-b221-94805f26ef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.539116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9b623f25-e431-4c0c-b221-94805f26ef54 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:13.540663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b623f25-e431-4c0c-b221-94805f26ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:13.584846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ecf51-3b87-46f2-8a3f-819592b31e0a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:13.590618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:13.608690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 19/39] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:13.610059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0018f-49a3-4aaf-ac82-e875f6e9760e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.610344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0018f-49a3-4aaf-ac82-e875f6e9760e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.610616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0018f-49a3-4aaf-ac82-e875f6e9760e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.611855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 20/40] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:13.612533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0018f-49a3-4aaf-ac82-e875f6e9760e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:13.638678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f5ed7-9770-42e0-b4f5-2325e30789e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:13.639265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5ed7-9770-42e0-b4f5-2325e30789e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.639609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5ed7-9770-42e0-b4f5-2325e30789e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.639889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5ed7-9770-42e0-b4f5-2325e30789e6 None admin] PATH_INFO: `/v3/domains/default/users/9242a7b2cdf946a38262cb2b38f5cb44/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.642320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f5ed7-9770-42e0-b4f5-2325e30789e6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9242a7b2cdf946a38262cb2b38f5cb44, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:13.643793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f5ed7-9770-42e0-b4f5-2325e30789e6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:13.666878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 21/41] 10.222.0.20 () {58 vars in 1361 bytes} [Tue Sep 13 13:58:13 2022] PUT /identity/v3/domains/default/users/9242a7b2cdf946a38262cb2b38f5cb44/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:58:13.668892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575070a-9ee2-4985-8c27-65cbb2266f34 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.669292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575070a-9ee2-4985-8c27-65cbb2266f34 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.669644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575070a-9ee2-4985-8c27-65cbb2266f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.701184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2575070a-9ee2-4985-8c27-65cbb2266f34 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:13.704112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2575070a-9ee2-4985-8c27-65cbb2266f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:13.736238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0018f-49a3-4aaf-ac82-e875f6e9760e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:13.793751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:13.793751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:13.793751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:13.793751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:13.794171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:13.794171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:13.822426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:13.831574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 21/43] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 13:58:13 2022] POST /identity/v3/projects => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:13.881303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575070a-9ee2-4985-8c27-65cbb2266f34 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:13.891134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:14.006858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575070a-9ee2-4985-8c27-65cbb2266f34 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:14.046480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:14.047110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:14.047392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:14.047667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:14.048347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:14.049748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:14.069945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 23/44] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 13 13:58:13 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:14.128435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:14.169146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:14.354772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabde62f-feca-4997-ab6f-483f6a98025b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:14.388627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9eef0-54b5-44f9-857d-67cdd1d6fe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:14.389260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9eef0-54b5-44f9-857d-67cdd1d6fe61 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:14.390220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9eef0-54b5-44f9-857d-67cdd1d6fe61 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:14.390498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9eef0-54b5-44f9-857d-67cdd1d6fe61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:14.391617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9eef0-54b5-44f9-857d-67cdd1d6fe61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:14.393069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9eef0-54b5-44f9-857d-67cdd1d6fe61 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:14.394733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 22/45] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:14.400603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:14.464652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:14.500918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d12e83-0135-4161-8c40-cb2ff75b8337 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:14.537274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f65e68-8044-495b-884a-e377bc90edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:14.537918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f65e68-8044-495b-884a-e377bc90edc9 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:14.538230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f65e68-8044-495b-884a-e377bc90edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:14.538505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f65e68-8044-495b-884a-e377bc90edc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:14.539164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f65e68-8044-495b-884a-e377bc90edc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:14.540605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f65e68-8044-495b-884a-e377bc90edc9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:14.568829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 13:58:14 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:15.941606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bde0-b885-4e0c-8854-f3b379a55585 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:15.942158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bde0-b885-4e0c-8854-f3b379a55585 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:15.942158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bde0-b885-4e0c-8854-f3b379a55585 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:15.943463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 23/47] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:15.948649 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d241ca-50ca-461a-b39d-28228a3e8296 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:15.949063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d241ca-50ca-461a-b39d-28228a3e8296 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:15.949204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d241ca-50ca-461a-b39d-28228a3e8296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:15.967459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d7d241ca-50ca-461a-b39d-28228a3e8296 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:15.969335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d7d241ca-50ca-461a-b39d-28228a3e8296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:16.032740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 25/48] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:16.302499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bde0-b885-4e0c-8854-f3b379a55585 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:16.402780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bde0-b885-4e0c-8854-f3b379a55585 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:16.405615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:16.437372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:16.438022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:16.438308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:16.438586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:16.448828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:16.453888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:16.454756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:16.455697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 24/49] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:16.460792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d241ca-50ca-461a-b39d-28228a3e8296 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:16.508171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:16.563088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d241ca-50ca-461a-b39d-28228a3e8296 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:16.596856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f1fd1-b08e-417c-a896-a8a255eed774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:16.597461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f1fd1-b08e-417c-a896-a8a255eed774 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:16.597765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f1fd1-b08e-417c-a896-a8a255eed774 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:16.598045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f1fd1-b08e-417c-a896-a8a255eed774 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:16.598703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f1fd1-b08e-417c-a896-a8a255eed774 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:16.599005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f1fd1-b08e-417c-a896-a8a255eed774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:16.600400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f1fd1-b08e-417c-a896-a8a255eed774 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:16.611667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 26/50] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:16.617575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:16.718574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a115d-f005-42aa-ab13-4c070637e9c9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:16.737718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7388-cb77-4036-b93f-f37205bfb557 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:16.738030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7388-cb77-4036-b93f-f37205bfb557 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:16.738309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7388-cb77-4036-b93f-f37205bfb557 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:16.739642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 27/51] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:16.744542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d62129-0099-4c0e-a20f-134f229b4b4e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:16.744831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d62129-0099-4c0e-a20f-134f229b4b4e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:16.745112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d62129-0099-4c0e-a20f-134f229b4b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:16.751268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:16.751902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:16.752187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:16.752462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] PATH_INFO: `/v3/users/9242a7b2cdf946a38262cb2b38f5cb44` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:16.753591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] RBAC: Authorizing `identity:get_user(user_id=9242a7b2cdf946a38262cb2b38f5cb44)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:16.755082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:16.756698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 25/52] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:16 2022] GET /identity/v3/users/9242a7b2cdf946a38262cb2b38f5cb44 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:16.762329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:16.796608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:16.866766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e6d62129-0099-4c0e-a20f-134f229b4b4e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:16.868007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6d62129-0099-4c0e-a20f-134f229b4b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:16.868581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b07e14-315f-480f-8c48-53ac088e69fb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:16.902518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:16.903157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:16.903439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:16.903715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:16.904838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:16.906954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:16.908531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:16.914197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:16.946271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 28/54] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:17.008345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:17.012192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f842-a1ff-4e78-af73-3e0f9aab5e6f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:17.045177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:17.046440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:17.046731 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:17.047007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:17.047729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.domain.id=default, user.id=9242a7b2cdf946a38262cb2b38f5cb44) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:17.048036 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:17.049482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:17.055050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Sep 13 13:58:16 2022] GET /identity/v3/role_assignments?user.id=9242a7b2cdf946a38262cb2b38f5cb44&scope.domain.id=default&role.id=96af379f1477489b8542698d3c012f1a => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:17.216146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d62129-0099-4c0e-a20f-134f229b4b4e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:17.259835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:17.317920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d62129-0099-4c0e-a20f-134f229b4b4e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:17.351126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793192a-56b2-44ce-bdfa-5e8f3787c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:17.351756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793192a-56b2-44ce-bdfa-5e8f3787c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:17.352039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793192a-56b2-44ce-bdfa-5e8f3787c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:17.352314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793192a-56b2-44ce-bdfa-5e8f3787c9a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:17.353413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793192a-56b2-44ce-bdfa-5e8f3787c9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:17.354895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793192a-56b2-44ce-bdfa-5e8f3787c9a9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:17.356456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 29/56] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:17.362300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:17.462340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf81d-c07c-4d3b-a99d-946733e6db7b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:17.496300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad8865b-13f0-48c4-93ce-9873829dd8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:17.496837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8865b-13f0-48c4-93ce-9873829dd8d8 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:17.497120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8865b-13f0-48c4-93ce-9873829dd8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:17.497397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad8865b-13f0-48c4-93ce-9873829dd8d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:17.498089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8865b-13f0-48c4-93ce-9873829dd8d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:17.499619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad8865b-13f0-48c4-93ce-9873829dd8d8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:17.527184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 28/57] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Sep 13 13:58:17 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:19.675403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efef863-bc49-41e8-999d-848520b2e6e5 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:19.675942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efef863-bc49-41e8-999d-848520b2e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:19.675942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efef863-bc49-41e8-999d-848520b2e6e5 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:19.677282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 30/58] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:19.682501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f74091-ce5e-430b-8915-6fd474977cb1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:19.682905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f74091-ce5e-430b-8915-6fd474977cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:19.683055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f74091-ce5e-430b-8915-6fd474977cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:19.701140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-51f74091-ce5e-430b-8915-6fd474977cb1 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:19.702424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-51f74091-ce5e-430b-8915-6fd474977cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:19.712773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:19.768236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 29/59] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:20.041289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efef863-bc49-41e8-999d-848520b2e6e5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:20.141408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efef863-bc49-41e8-999d-848520b2e6e5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:20.150666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:20.175728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff534ef-7fb5-4ddf-a06b-f56109036617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:20.176349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff534ef-7fb5-4ddf-a06b-f56109036617 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:20.176631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff534ef-7fb5-4ddf-a06b-f56109036617 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:20.176906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff534ef-7fb5-4ddf-a06b-f56109036617 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:20.178031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff534ef-7fb5-4ddf-a06b-f56109036617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:20.179461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff534ef-7fb5-4ddf-a06b-f56109036617 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:20.181009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 31/60] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:20.186612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f74091-ce5e-430b-8915-6fd474977cb1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:20.283373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f74091-ce5e-430b-8915-6fd474977cb1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:20.316134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd5d0b-2201-49e3-a121-fac2e9ca87c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:20.316674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5d0b-2201-49e3-a121-fac2e9ca87c0 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:20.316961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5d0b-2201-49e3-a121-fac2e9ca87c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:20.317242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd5d0b-2201-49e3-a121-fac2e9ca87c0 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:20.317931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd5d0b-2201-49e3-a121-fac2e9ca87c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:20.319383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd5d0b-2201-49e3-a121-fac2e9ca87c0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:20.345860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 30/61] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:58:20 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:23.630179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce121c-840c-4d5c-83c9-9e5c44c27c91 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:23.631187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce121c-840c-4d5c-83c9-9e5c44c27c91 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:23.631711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce121c-840c-4d5c-83c9-9e5c44c27c91 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:23.633512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 32/62] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:23.638498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fcb47-5f0e-485f-8634-60ce10f4238c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:23.638900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fcb47-5f0e-485f-8634-60ce10f4238c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:23.639066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fcb47-5f0e-485f-8634-60ce10f4238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:23.645816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:23.660843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ee3fcb47-5f0e-485f-8634-60ce10f4238c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:23.666391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ee3fcb47-5f0e-485f-8634-60ce10f4238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:23.691654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73027bb8-740b-4790-bcb2-a9e775a19ddf None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:23.692280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73027bb8-740b-4790-bcb2-a9e775a19ddf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:23.692790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73027bb8-740b-4790-bcb2-a9e775a19ddf None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:23.694355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 33/63] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:23.700068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1bc76-bd9b-4cfd-82af-d3bcce59ba88 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:23.700588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1bc76-bd9b-4cfd-82af-d3bcce59ba88 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:23.701083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1bc76-bd9b-4cfd-82af-d3bcce59ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:23.739178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fda1bc76-bd9b-4cfd-82af-d3bcce59ba88 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:23.740378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fda1bc76-bd9b-4cfd-82af-d3bcce59ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:23.772192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 31/64] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:23.847320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:24.134821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fcb47-5f0e-485f-8634-60ce10f4238c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.146072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968171b-9c44-4345-a7de-becd219c6adb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.148407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968171b-9c44-4345-a7de-becd219c6adb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.148886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968171b-9c44-4345-a7de-becd219c6adb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.151827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 35/66] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:24.157510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469976-3477-41f1-9988-7b0426d56daa None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.157857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469976-3477-41f1-9988-7b0426d56daa None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469976-3477-41f1-9988-7b0426d56daa None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.160195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 36/67] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:24.161527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82af7-fba7-4d12-94f1-f9c77dcb34fe None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.162042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82af7-fba7-4d12-94f1-f9c77dcb34fe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.162514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d82af7-fba7-4d12-94f1-f9c77dcb34fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.182449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d6d82af7-fba7-4d12-94f1-f9c77dcb34fe None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:24.184043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d6d82af7-fba7-4d12-94f1-f9c77dcb34fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:24.254182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fcb47-5f0e-485f-8634-60ce10f4238c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:24.261939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 37/68] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:24.261939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63438d-f08c-4241-9e1c-370fc3268ba6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.261939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63438d-f08c-4241-9e1c-370fc3268ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.261939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63438d-f08c-4241-9e1c-370fc3268ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.282419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cf63438d-f08c-4241-9e1c-370fc3268ba6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:24.283673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf63438d-f08c-4241-9e1c-370fc3268ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:24.289758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:24.296518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:24.297334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.297905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.298358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.310997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:24.317364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:24.318486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:24.319481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:24.320922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:24.323026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 32/69] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:24.324983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.370595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 38/70] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:24.371461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63438d-f08c-4241-9e1c-370fc3268ba6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.460895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8047d-8182-48f6-ab39-0402f88b3f42 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:24.501478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6f72a-0969-4212-9a9a-d60555a188e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:24.502581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f72a-0969-4212-9a9a-d60555a188e8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.503053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f72a-0969-4212-9a9a-d60555a188e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.503528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f72a-0969-4212-9a9a-d60555a188e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.512287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63438d-f08c-4241-9e1c-370fc3268ba6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:24.515587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6f72a-0969-4212-9a9a-d60555a188e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fed6f72a-0969-4212-9a9a-d60555a188e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:24.521034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:24.521930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:24.522737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:24.525435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 33/71] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:24.531759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216082dc-851d-4d40-99c8-352b76a181b8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.532166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216082dc-851d-4d40-99c8-352b76a181b8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.532553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216082dc-851d-4d40-99c8-352b76a181b8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.534679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 34/72] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:24.539863 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216082dc-851d-4d40-99c8-352b76a181b8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.567046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6d5f8-1a27-4d10-baad-d992f59f8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:24.567687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6d5f8-1a27-4d10-baad-d992f59f8f6a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.567966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6d5f8-1a27-4d10-baad-d992f59f8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.568237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6d5f8-1a27-4d10-baad-d992f59f8f6a None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.568900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6d5f8-1a27-4d10-baad-d992f59f8f6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:24.569204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6d5f8-1a27-4d10-baad-d992f59f8f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:24.570696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6d5f8-1a27-4d10-baad-d992f59f8f6a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:24.579632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 39/73] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:24.585414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b726f-bd68-4f5c-bedd-660772961be2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.585708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b726f-bd68-4f5c-bedd-660772961be2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.586402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b726f-bd68-4f5c-bedd-660772961be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.598021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:24.607074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4a6b726f-bd68-4f5c-bedd-660772961be2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:24.608175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4a6b726f-bd68-4f5c-bedd-660772961be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:24.634235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:24.654397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216082dc-851d-4d40-99c8-352b76a181b8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:24.682236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 40/74] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:24.683078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b726f-bd68-4f5c-bedd-660772961be2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.688641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:24.689256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.689538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.689843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.690516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:24.690851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:24.692241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:24.700709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 35/75] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:24.706843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.788190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b726f-bd68-4f5c-bedd-660772961be2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:24.813270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1feee7-5e06-47b1-8f85-ac4a45dbcdcc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:24.824966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:24.825601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.825916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.826200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.828044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:24.839004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:24.845429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:24.846317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:24.847074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 41/76] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:24.851811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.853158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:24.853836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:24.854129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:24.854412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:24.868912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:24.870415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:24.872009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:24.872119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 36/77] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:24.872907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:24.970614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736417fa-26c0-4f7b-9fca-04e9fe99ff0f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:24.981022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931575c-9e1e-48d8-aa1a-263868716211 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.004088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.004721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.005002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.005277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.015752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:25.017997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.018622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.018904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.019178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] PATH_INFO: `/v3/users/9242a7b2cdf946a38262cb2b38f5cb44` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.020316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] RBAC: Authorizing `identity:get_user(user_id=9242a7b2cdf946a38262cb2b38f5cb44)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:25.020799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:25.021662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:25.022523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:25.023459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.023527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 42/78] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:25.023527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.023648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 37/79] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:24 2022] GET /identity/v3/users/9242a7b2cdf946a38262cb2b38f5cb44 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:25.024112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.067517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.105518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.127164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa680c-d2f0-400c-bdb7-350c44c799e3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.127622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ca04-3770-4b2f-a0ea-217962c59d64 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.160987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.161592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.161697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.161917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.162194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.162305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.162594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.162875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.163549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:25.163860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.166173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.175292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 43/80] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:25.175292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.234308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.235904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.273623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84467c-fcb0-409a-a9c0-90c3081c2e7f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.275166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.306917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.307551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.307836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.308116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.309238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.310730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.312965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 44/81] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:25.313745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.328547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.337396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 38/82] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:25.338280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.427274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de259f-6ed6-4f18-a344-10a5d0bfebd1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.440990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8608aa-995c-4b6d-ab12-4278f453f5bd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.459998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.460611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.460897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.461173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.461875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:25.462185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.463550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.472521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 45/83] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:25.473320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.474355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.474968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.475251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.475524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.487133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.546530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.546698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.548468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.578237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d350cfb-ef2c-49bd-a59b-84048e307a7f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.612623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.613186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.613468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.613765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.614888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.616325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.617952 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 46/84] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:25.622656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.654051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 39/85] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:25.654051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.690626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.724446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d427418-e574-4e89-bd90-d3b8c1965004 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.756980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.757582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.757900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.758177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.758826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.760272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.768099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8f71-4d5e-4965-b243-23f19a57a93f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.775842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.789678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 47/86] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:58:25 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:25.790520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.812165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.812987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.813276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.813586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.815493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.817013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.818878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 40/87] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:25.824910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.907582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4c6c9-d184-43c8-91a0-31d4a0134c74 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.936795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e9675-01bf-429a-bfc2-f47f7f60c3ee None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:25.947273 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.947489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:25.947992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.948362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.948622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:25.950084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:25.951575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:25.953633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 48/88] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:25.954462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:25.984408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:25.985194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:25.985479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:25.985836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:26.027365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:26.069208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:26.072291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:26.083013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088183e-d67c-4048-bac0-87f3072c89b4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:26.127196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:26.128216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:26.128685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:26.129166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:26.130691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:26.132961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:26.134949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 49/89] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:26.135905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:26.199543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 41/90] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:26.200427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:26.212080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:26.271998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4197a4-5566-4598-a737-af8397a39080 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:26.309442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:26.313241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4741d1-80ea-4921-a6ee-4d42a16687a9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:26.318011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:26.318632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:26.318911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:26.319185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:26.319920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=dda66a8495034a4f94e940f3175970f3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:26.320227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:26.321667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:26.337750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 50/91] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:26 2022] GET /identity/v3/role_assignments?user.id=dda66a8495034a4f94e940f3175970f3&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:26.337750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:26.383434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:26.384064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:26.384344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:26.384619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:26.497040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:26.498500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:26.515487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:26.544466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcbc3bd-c4c2-42a3-a737-96c465af190f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:26.608097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:26.609272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:26.609554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:26.609976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:26.610717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=9242a7b2cdf946a38262cb2b38f5cb44) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:26.611134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:26.612634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:26.618103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 42/92] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:26.619221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:26.628783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 51/93] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:26 2022] GET /identity/v3/role_assignments?user.id=9242a7b2cdf946a38262cb2b38f5cb44&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:26.629470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:26.786254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25111da-9122-4ace-a85a-e7c3329dcd5c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:26.794169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:26.818259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:26.902568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:26.903235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:26.903525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:26.903805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:26.905083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:26.907505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:26.909240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 43/94] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:26 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:26.915103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:26.934030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e888ed-ce13-42c0-9b1c-f327fc7f505d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:26.996976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:26.997625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:26.997935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:26.998201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] PATH_INFO: `/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:27.001112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] RBAC: Authorizing `identity:get_project(project_id=d7ec4926fdb8498bac0fa968d08a6b3d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:27.004691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:27.004691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 52/95] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:26 2022] GET /identity/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d => generated 312 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:27.015181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:27.062117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36643af-5e64-4af2-ade5-532c2bafa4c1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:27.121660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8543bca-1da3-4f8d-b330-5672cfe6070c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:27.143376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cd4a0-b459-46d7-adc9-e9fed619fc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:27.144004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cd4a0-b459-46d7-adc9-e9fed619fc4e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:27.144289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cd4a0-b459-46d7-adc9-e9fed619fc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:27.144567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cd4a0-b459-46d7-adc9-e9fed619fc4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:27.145309 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cd4a0-b459-46d7-adc9-e9fed619fc4e None admin] RBAC: Adding query filter params (role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=dda66a8495034a4f94e940f3175970f3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:27.145611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cd4a0-b459-46d7-adc9-e9fed619fc4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:27.147159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cd4a0-b459-46d7-adc9-e9fed619fc4e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:27.156283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297d67b-127b-431a-8b28-d7e9b39d110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:27.156896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297d67b-127b-431a-8b28-d7e9b39d110c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:27.157182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297d67b-127b-431a-8b28-d7e9b39d110c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:27.157463 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297d67b-127b-431a-8b28-d7e9b39d110c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:27.158223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297d67b-127b-431a-8b28-d7e9b39d110c None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=d7ec4926fdb8498bac0fa968d08a6b3d, user.id=dda66a8495034a4f94e940f3175970f3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:27.158541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297d67b-127b-431a-8b28-d7e9b39d110c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:27.160004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297d67b-127b-431a-8b28-d7e9b39d110c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:27.161658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 44/96] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:26 2022] GET /identity/v3/role_assignments?user.id=dda66a8495034a4f94e940f3175970f3&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:27.162176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:27.167677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 53/97] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:27 2022] GET /identity/v3/role_assignments?user.id=dda66a8495034a4f94e940f3175970f3&scope.project.id=d7ec4926fdb8498bac0fa968d08a6b3d&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:28.564430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65fe9-216a-49d7-b96f-8d35467e660f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:28.564926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65fe9-216a-49d7-b96f-8d35467e660f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:28.564998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65fe9-216a-49d7-b96f-8d35467e660f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:28.566357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 45/98] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:28.583680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ef9f-790f-4fde-a926-0bb54b0a96d2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:28.584205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ef9f-790f-4fde-a926-0bb54b0a96d2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:28.584466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ef9f-790f-4fde-a926-0bb54b0a96d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:28.610949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0221ef9f-790f-4fde-a926-0bb54b0a96d2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:28.612255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0221ef9f-790f-4fde-a926-0bb54b0a96d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:28.689309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 54/99] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:28.714930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b0cf9-c619-4362-ba5e-539e1814a1bd None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:28.715268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b0cf9-c619-4362-ba5e-539e1814a1bd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:28.715556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b0cf9-c619-4362-ba5e-539e1814a1bd None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:28.717083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 46/100] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:28.722435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ec9b-e12d-4bc8-abe0-e1d3b3e08f06 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:28.722723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ec9b-e12d-4bc8-abe0-e1d3b3e08f06 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:28.723001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ec9b-e12d-4bc8-abe0-e1d3b3e08f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:28.742178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-76a1ec9b-e12d-4bc8-abe0-e1d3b3e08f06 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:28.743370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-76a1ec9b-e12d-4bc8-abe0-e1d3b3e08f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:28.832093 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 55/101] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:29.275634 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b0cf9-c619-4362-ba5e-539e1814a1bd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:29.291509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459f86e-265a-486b-bbae-318297886f56 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:29.295328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459f86e-265a-486b-bbae-318297886f56 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:29.295813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459f86e-265a-486b-bbae-318297886f56 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:29.299378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 56/102] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:29.303172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1f516-8727-4249-981e-9c5f6353604d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:29.303661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1f516-8727-4249-981e-9c5f6353604d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:29.304129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1f516-8727-4249-981e-9c5f6353604d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:29.339687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-aab1f516-8727-4249-981e-9c5f6353604d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:29.345549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-aab1f516-8727-4249-981e-9c5f6353604d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:29.357934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:29.419714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:29.460434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b0cf9-c619-4362-ba5e-539e1814a1bd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:29.463812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 57/103] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:29.465894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1f516-8727-4249-981e-9c5f6353604d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:29.532401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:29.537748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:29.537748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:29.537748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:29.537748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:29.539861 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:29.543647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 47/104] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:29.562897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:29.684487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1f516-8727-4249-981e-9c5f6353604d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:29.752500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:29.753153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:29.753443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:29.753720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:29.760333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57349188-ea08-452c-b20f-5e433865a5ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:29.777454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:29.782250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:29.790433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:29.791396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:29.792167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:29.794440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 58/105] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:29.802291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:29.816794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:29.817831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:29.818316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:29.818823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:29.819789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:29.821571 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:29.865766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 48/106] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 13:58:29 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:29.873032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:29.947940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6c72-2237-404e-a706-ab42615b2080 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:29.986278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:29.986896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:29.987182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:29.987506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:29.988196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:29.988548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:29.989974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:30.000009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 59/107] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:30.006225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:30.082933 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:30.130868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c90e0-de47-4070-b1f2-89a61de750f0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:30.179162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:30.179984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.180271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.180554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] PATH_INFO: `/v3/users/b52a0d8269c24af3ad837563e11e4fdd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.204535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] RBAC: Authorizing `identity:get_user(user_id=b52a0d8269c24af3ad837563e11e4fdd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:30.206407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:30.208494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 49/108] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:30 2022] GET /identity/v3/users/b52a0d8269c24af3ad837563e11e4fdd => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:30.215789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:30.266117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:30.307209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d1b55-809a-4d1f-bca8-ca44f453875a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:30.309170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:30.380209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2fedd-4237-46b3-8243-b7bf2c2583fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:30.381491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2fedd-4237-46b3-8243-b7bf2c2583fa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.382008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2fedd-4237-46b3-8243-b7bf2c2583fa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.382481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2fedd-4237-46b3-8243-b7bf2c2583fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.393393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cc343-7a59-4c94-9795-057264caf42a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:30.410181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2fedd-4237-46b3-8243-b7bf2c2583fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4ad2fedd-4237-46b3-8243-b7bf2c2583fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:30.416753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:30.417583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:30.418398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:30.419492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 60/109] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:30.419889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8488d-91ff-430a-b092-430082bdb987 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.420139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8488d-91ff-430a-b092-430082bdb987 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.420418 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8488d-91ff-430a-b092-430082bdb987 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.422072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 61/110] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:30.427204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf93266-9a40-4d36-a69b-8f1800c3ac95 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.427718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf93266-9a40-4d36-a69b-8f1800c3ac95 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.428012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf93266-9a40-4d36-a69b-8f1800c3ac95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.439540 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:30.440166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.440452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.440734 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.456610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bdf93266-9a40-4d36-a69b-8f1800c3ac95 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:30.458910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bdf93266-9a40-4d36-a69b-8f1800c3ac95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:30.517520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:30.518946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:30.542527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 62/111] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:30.544054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf93266-9a40-4d36-a69b-8f1800c3ac95 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:30.550198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:30.654779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:30.666783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf93266-9a40-4d36-a69b-8f1800c3ac95 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:30.705776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a656f-2baa-4962-bea7-d1b1ebf18895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:30.707195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a656f-2baa-4962-bea7-d1b1ebf18895 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.707758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a656f-2baa-4962-bea7-d1b1ebf18895 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.708228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a656f-2baa-4962-bea7-d1b1ebf18895 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.709099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a656f-2baa-4962-bea7-d1b1ebf18895 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:30.709596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a656f-2baa-4962-bea7-d1b1ebf18895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:30.711297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a656f-2baa-4962-bea7-d1b1ebf18895 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:30.720465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 63/112] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:30.722003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fbe66-3593-4398-887f-72041096beec None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.722554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fbe66-3593-4398-887f-72041096beec None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.723024 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fbe66-3593-4398-887f-72041096beec None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.724606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 64/113] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:30.731092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea985a6-2d2b-4c2b-b228-4a4e552052ae None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.731603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea985a6-2d2b-4c2b-b228-4a4e552052ae None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.732120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea985a6-2d2b-4c2b-b228-4a4e552052ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.755166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 50/114] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:30.755166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:30.756917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1ea985a6-2d2b-4c2b-b228-4a4e552052ae None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:30.758375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1ea985a6-2d2b-4c2b-b228-4a4e552052ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:30.831943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 65/115] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:30.833265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea985a6-2d2b-4c2b-b228-4a4e552052ae None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:30.928098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2f042-e41a-469d-8b7e-1266e7ff4413 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:30.934916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:30.947897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:30.955929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea985a6-2d2b-4c2b-b228-4a4e552052ae None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:30.968345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e830f-a384-4a6b-87d1-e41d1404fd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:30.969331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e830f-a384-4a6b-87d1-e41d1404fd1f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.969628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e830f-a384-4a6b-87d1-e41d1404fd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.969973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e830f-a384-4a6b-87d1-e41d1404fd1f None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.971164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e830f-a384-4a6b-87d1-e41d1404fd1f None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:30.972733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e830f-a384-4a6b-87d1-e41d1404fd1f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:30.974475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 51/116] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:30 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:30.975625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659544a5-7380-409a-ba97-2dd3aed0aeb4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:30.975909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659544a5-7380-409a-ba97-2dd3aed0aeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:30.976232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659544a5-7380-409a-ba97-2dd3aed0aeb4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:30.977692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 52/117] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:30.983188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659544a5-7380-409a-ba97-2dd3aed0aeb4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.000757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.001806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.002287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.002759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.004097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:31.006227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:31.008188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 66/118] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:30 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:31.009686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.101508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659544a5-7380-409a-ba97-2dd3aed0aeb4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.123220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25c695-5ad7-4aaf-8ae1-0b628dac5ea1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.141748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781f5e3-cfad-4198-ae34-54641762c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.142577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781f5e3-cfad-4198-ae34-54641762c986 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.142937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781f5e3-cfad-4198-ae34-54641762c986 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.143295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781f5e3-cfad-4198-ae34-54641762c986 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.154493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781f5e3-cfad-4198-ae34-54641762c986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d781f5e3-cfad-4198-ae34-54641762c986 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:31.160489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:31.161384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 53/119] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:31.166413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e9c2c-e062-4228-ba9b-c04bf29f76c7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.167351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e9c2c-e062-4228-ba9b-c04bf29f76c7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.167351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e9c2c-e062-4228-ba9b-c04bf29f76c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.169117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.169781 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.170217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.170516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.172048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:31.173496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:31.175330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 67/120] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:31 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:31.176360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.184455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3c9e9c2c-e062-4228-ba9b-c04bf29f76c7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:31.185621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3c9e9c2c-e062-4228-ba9b-c04bf29f76c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:31.200860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:31.232088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:31.260111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 54/121] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:31.261141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e9c2c-e062-4228-ba9b-c04bf29f76c7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.294186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1a3d5-c497-4829-90e8-ab790b2d7444 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.342067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.343353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.343986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.344336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.345719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] RBAC: Adding query filter params (role.id=969ed0ca7dc147ecad313e6a3909656d, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=b52a0d8269c24af3ad837563e11e4fdd) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:31.346149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:31.347973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:31.355372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 68/122] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:31 2022] GET /identity/v3/role_assignments?user.id=b52a0d8269c24af3ad837563e11e4fdd&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=969ed0ca7dc147ecad313e6a3909656d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:31.356262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.393565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e9c2c-e062-4228-ba9b-c04bf29f76c7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.475842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.476503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.476785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.481098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.508523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:31.515834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:31.516913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:31.517688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:31.519876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 55/123] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:58:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:31.522943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.540140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:31.566847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f891-f911-402d-95ae-f92db214bdfa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.580379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:31.605887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.606828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.607236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.607657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.608328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:31.609172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:31.610563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:31.620427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 69/124] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:31.621259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.664236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae05b17-0480-46de-98ae-e4ae7458294f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.712017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.713213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.713505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.713812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955/users/dda66a8495034a4f94e940f3175970f3/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.716678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] RBAC: Authorizing `identity:create_grant(project_id=18b7b138b42148d79097c7b2e243b955, user_id=dda66a8495034a4f94e940f3175970f3, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:31.719234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:31.741038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 56/125] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:31 2022] PUT /identity/v3/projects/18b7b138b42148d79097c7b2e243b955/users/dda66a8495034a4f94e940f3175970f3/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:31.741298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c883f8-c965-4207-a3af-4bca9b375548 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.741803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.784377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.784994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.785310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.785642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.786409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:31.786719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:31.788161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:31.798595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 70/126] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:58:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:31.799718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:31.812492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:31.882692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:31.893334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe7ce-0080-4fc0-9491-db4ab4d04219 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.942700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551ce78-cd56-441e-b34b-408f03d057c9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:31.963818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:31.965288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:31.965570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:31.965873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:31.989434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:31.997227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:31.998146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:31.998959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:32.002168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 57/127] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:32.003085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.010006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.010655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.010938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.011212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] PATH_INFO: `/v3/users/9242a7b2cdf946a38262cb2b38f5cb44` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.012366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] RBAC: Authorizing `identity:get_user(user_id=9242a7b2cdf946a38262cb2b38f5cb44)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.013942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.015694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 71/128] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:31 2022] GET /identity/v3/users/9242a7b2cdf946a38262cb2b38f5cb44 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:32.016528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.126361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:32.128420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828ab6e-ddd6-4d00-bcfa-46353c003c4c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:32.157605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6509bb-dc20-403b-9187-5d5194123dac None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:32.194523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.195145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.195427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.195868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.196548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:32.196875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.198594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.207746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 72/129] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:32.208683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.230469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.231618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.232108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.232586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.238460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.240210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.242075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 58/130] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:32 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:32.243055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.314197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:32.320079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c68bed-c3d3-4cc0-9712-1479f78c3d9f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:32.358402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.359017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.359306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.359712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.360998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.362478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.364111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 73/131] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:32 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:32.364958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.374420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:32.479118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30690e5e-b9fc-4631-bbda-417f745af83e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:32.548746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.549812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.550301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.550777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.552740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.554521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.556592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 59/132] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:32 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:32.557943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.611798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef8828-5262-44cd-836f-8fff95312e18 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:32.679281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:32.685770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83292cf6-1e76-40d5-b896-d9150f670f3f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:32.705152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.705817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.706103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.706381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.708147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.709674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.713300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 74/133] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:32 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:32.714050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.743961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20868ad-d496-4b6f-92bc-ce8bd50e3f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.745237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20868ad-d496-4b6f-92bc-ce8bd50e3f3e None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.745994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20868ad-d496-4b6f-92bc-ce8bd50e3f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.746293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20868ad-d496-4b6f-92bc-ce8bd50e3f3e None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955/users/9242a7b2cdf946a38262cb2b38f5cb44/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.749023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20868ad-d496-4b6f-92bc-ce8bd50e3f3e None admin] RBAC: Authorizing `identity:create_grant(project_id=18b7b138b42148d79097c7b2e243b955, user_id=9242a7b2cdf946a38262cb2b38f5cb44, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.750543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20868ad-d496-4b6f-92bc-ce8bd50e3f3e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.765883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 60/134] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:32 2022] PUT /identity/v3/projects/18b7b138b42148d79097c7b2e243b955/users/9242a7b2cdf946a38262cb2b38f5cb44/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:32.767260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c133f-ca5a-4d5e-9d7e-e5faefdf7d49 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.767561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c133f-ca5a-4d5e-9d7e-e5faefdf7d49 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.767839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c133f-ca5a-4d5e-9d7e-e5faefdf7d49 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.773601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 61/135] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:32.773601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c133f-ca5a-4d5e-9d7e-e5faefdf7d49 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:32.810842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:32.839867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fe8e-296a-459f-bf85-068c427eb040 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:32.882848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc6d4e-ce1c-483a-9872-bf786a51aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:32.883796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc6d4e-ce1c-483a-9872-bf786a51aadf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.884249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc6d4e-ce1c-483a-9872-bf786a51aadf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.884705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc6d4e-ce1c-483a-9872-bf786a51aadf None admin] PATH_INFO: `/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.886807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc6d4e-ce1c-483a-9872-bf786a51aadf None admin] RBAC: Authorizing `identity:get_project(project_id=d7ec4926fdb8498bac0fa968d08a6b3d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:32.888508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc6d4e-ce1c-483a-9872-bf786a51aadf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:32.891087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 75/136] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:32 2022] GET /identity/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:32.897315 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d277c-cbe2-4a15-8b6a-23f927634b73 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:32.898552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d277c-cbe2-4a15-8b6a-23f927634b73 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:32.899037 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d277c-cbe2-4a15-8b6a-23f927634b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:32.919920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-289d277c-cbe2-4a15-8b6a-23f927634b73 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:32.921577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-289d277c-cbe2-4a15-8b6a-23f927634b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:32.926437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c133f-ca5a-4d5e-9d7e-e5faefdf7d49 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:33.005145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 76/137] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:33.006443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d277c-cbe2-4a15-8b6a-23f927634b73 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:33.007133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:33.007811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:33.008096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:33.008373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955/users/dda66a8495034a4f94e940f3175970f3/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:33.010847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] RBAC: Authorizing `identity:create_grant(project_id=18b7b138b42148d79097c7b2e243b955, user_id=dda66a8495034a4f94e940f3175970f3, role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:33.013251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:33.030545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 62/138] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:32 2022] PUT /identity/v3/projects/18b7b138b42148d79097c7b2e243b955/users/dda66a8495034a4f94e940f3175970f3/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:33.093197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:33.128005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d277c-cbe2-4a15-8b6a-23f927634b73 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:33.178521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:33.179428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:33.179866 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:33.180276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] PATH_INFO: `/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d/users/dda66a8495034a4f94e940f3175970f3/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:33.182436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] RBAC: Authorizing `identity:create_grant(project_id=d7ec4926fdb8498bac0fa968d08a6b3d, user_id=dda66a8495034a4f94e940f3175970f3, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:33.184110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:33.203412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 77/139] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:33 2022] PUT /identity/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d/users/dda66a8495034a4f94e940f3175970f3/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:33.523625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:33.544107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:33.636322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854b7d5-740f-4e82-9392-a635a294bc41 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:33.677219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba9b9c-98fc-41ca-8433-172cd027c319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:33.677896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba9b9c-98fc-41ca-8433-172cd027c319 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:33.678185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba9b9c-98fc-41ca-8433-172cd027c319 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:33.678461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba9b9c-98fc-41ca-8433-172cd027c319 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:33.679598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba9b9c-98fc-41ca-8433-172cd027c319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:33.681051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba9b9c-98fc-41ca-8433-172cd027c319 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:33.682648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 63/140] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:33.688625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:33.833833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b435de7-d9c3-4b29-85cd-a9489597a75e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:33.872723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:33.879092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07ad17-8ab8-4ccb-a6d5-a723666e81b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:33.880023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07ad17-8ab8-4ccb-a6d5-a723666e81b3 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:33.880526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07ad17-8ab8-4ccb-a6d5-a723666e81b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:33.881038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07ad17-8ab8-4ccb-a6d5-a723666e81b3 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:33.881915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07ad17-8ab8-4ccb-a6d5-a723666e81b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:33.883579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07ad17-8ab8-4ccb-a6d5-a723666e81b3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:33.912041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 78/141] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:58:33 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:34.179422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e4da-eb1a-4c07-b9ab-d181605069f9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:34.179934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e4da-eb1a-4c07-b9ab-d181605069f9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:34.179934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e4da-eb1a-4c07-b9ab-d181605069f9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:34.181363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 64/142] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:34.188020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469647-9afb-4e2a-9afc-d9a60dc38136 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:34.188300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469647-9afb-4e2a-9afc-d9a60dc38136 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:34.188570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71469647-9afb-4e2a-9afc-d9a60dc38136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:34.219221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-71469647-9afb-4e2a-9afc-d9a60dc38136 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:34.220409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-71469647-9afb-4e2a-9afc-d9a60dc38136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:34.302465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 79/143] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:34.551693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889cdd-4946-4399-bb0a-ff1cd893f38d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:34.552012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889cdd-4946-4399-bb0a-ff1cd893f38d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:34.552257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99889cdd-4946-4399-bb0a-ff1cd893f38d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:34.553593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 65/144] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:34.559588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2a1ca-12d7-4f1e-a7c7-0f73c32108ca None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:34.559871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2a1ca-12d7-4f1e-a7c7-0f73c32108ca None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:34.560143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2a1ca-12d7-4f1e-a7c7-0f73c32108ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:34.585150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-46d2a1ca-12d7-4f1e-a7c7-0f73c32108ca None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:34.586377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-46d2a1ca-12d7-4f1e-a7c7-0f73c32108ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:34.668805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 80/145] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:34.712076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889cdd-4946-4399-bb0a-ff1cd893f38d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:34.809422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:34.834588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99889cdd-4946-4399-bb0a-ff1cd893f38d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:34.875378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:34.876055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:34.876339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:34.876614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:34.890533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:34.895461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:34.896362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:34.897130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 66/146] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:34.903467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2a1ca-12d7-4f1e-a7c7-0f73c32108ca None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:34.959317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:35.018242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:35.034317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2a1ca-12d7-4f1e-a7c7-0f73c32108ca None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:35.070739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:35.071269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:35.071546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:35.071815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:35.072484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:35.072789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:35.074211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:35.083670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 81/147] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:35.091494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:35.196700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aab856-9351-4297-b545-ed960d10bbb9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:35.214121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:35.234314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:35.235395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:35.235814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:35.236275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] PATH_INFO: `/v3/users/b52a0d8269c24af3ad837563e11e4fdd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:35.237571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] RBAC: Authorizing `identity:get_user(user_id=b52a0d8269c24af3ad837563e11e4fdd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:35.239255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:35.239715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93eb37-1ba2-4a13-b79d-073ec6651fdf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:35.241628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 82/148] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:35 2022] GET /identity/v3/users/b52a0d8269c24af3ad837563e11e4fdd => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:35.250216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:35.324569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:35.325294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:35.325684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:35.326021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:35.344220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:35.353776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:35.359117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:35.360499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:35.361268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 67/149] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:35.366303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:35.424659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761bc0-98c2-4482-9382-b817b761fbf9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:35.516823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:35.517465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:35.517756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:35.518036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:35.519179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:35.535031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc828bd3-9bd8-48e5-b5e5-0d90fcdfccd9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:35.536532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:35.538578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 83/150] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:35 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:35.544801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:35.590620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:35.591176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:35.591465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:35.591744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:35.592417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:35.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:35.594582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:35.617953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 68/151] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:35.630122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:35.632094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:35.748022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9cec4-a008-45bb-b829-c248c6a73267 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:35.767618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:35.802427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b47e3-d14e-4910-beed-89079ee1dc70 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:35.854543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:35.855270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:35.855574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:35.855850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:35.857047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:35.858830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:35.860790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 69/152] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:35 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:35.866814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:35.886477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:35.887170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:35.887455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:35.887731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955/users/b52a0d8269c24af3ad837563e11e4fdd/roles/969ed0ca7dc147ecad313e6a3909656d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:35.894531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] RBAC: Authorizing `identity:create_grant(project_id=18b7b138b42148d79097c7b2e243b955, user_id=b52a0d8269c24af3ad837563e11e4fdd, role_id=969ed0ca7dc147ecad313e6a3909656d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:35.896040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:35.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 84/153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:35 2022] PUT /identity/v3/projects/18b7b138b42148d79097c7b2e243b955/users/b52a0d8269c24af3ad837563e11e4fdd/roles/969ed0ca7dc147ecad313e6a3909656d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:58:35.982511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87df12d-a525-418e-85a6-e893c57a3131 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:36.014405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:36.022010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:36.022630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:36.022912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:36.023188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:36.097246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:36.098802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:36.207554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 70/154] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 13:58:36.213412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:36.364013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3370f746-677b-4e78-9a65-fec17f4fb887 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:36.377235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:36.415055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507671ec-f005-4ce5-a81b-2b0a9db07816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:36.415708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507671ec-f005-4ce5-a81b-2b0a9db07816 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:36.415992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507671ec-f005-4ce5-a81b-2b0a9db07816 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:36.416267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507671ec-f005-4ce5-a81b-2b0a9db07816 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:36.417437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507671ec-f005-4ce5-a81b-2b0a9db07816 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:36.418932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507671ec-f005-4ce5-a81b-2b0a9db07816 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:36.421145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 85/155] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:36 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:36.426915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:36.469563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:36.561134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57e5ee-0e7c-40e0-a580-b15566006e2b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:36.611915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1553a-79e3-4ed8-8973-594772c30b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:36.612536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1553a-79e3-4ed8-8973-594772c30b5b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:36.612849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1553a-79e3-4ed8-8973-594772c30b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:36.613077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1553a-79e3-4ed8-8973-594772c30b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:36.613849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1553a-79e3-4ed8-8973-594772c30b5b None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=dda66a8495034a4f94e940f3175970f3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:36.614159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1553a-79e3-4ed8-8973-594772c30b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:36.615601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1553a-79e3-4ed8-8973-594772c30b5b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:36.622066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 71/156] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:36 2022] GET /identity/v3/role_assignments?user.id=dda66a8495034a4f94e940f3175970f3&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:36.821044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86cb74-51a5-41bb-8009-9f4e87ba2329 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:36.821748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86cb74-51a5-41bb-8009-9f4e87ba2329 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:36.822268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86cb74-51a5-41bb-8009-9f4e87ba2329 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:36.823845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 86/157] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:36.848820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e8975-385f-44d4-8d94-abcd90397ff4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:36.849106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e8975-385f-44d4-8d94-abcd90397ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:36.849568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e8975-385f-44d4-8d94-abcd90397ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:36.878751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-199e8975-385f-44d4-8d94-abcd90397ff4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:36.880223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-199e8975-385f-44d4-8d94-abcd90397ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:36.964337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 72/158] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:37.067030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1aeb0c-6608-4716-b541-2046eeca0068 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:37.068035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1aeb0c-6608-4716-b541-2046eeca0068 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:37.068595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1aeb0c-6608-4716-b541-2046eeca0068 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:37.070188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 87/159] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:37.087878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4bb6a-0104-40c5-bcd3-1dc2f39df6ec None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:37.088153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4bb6a-0104-40c5-bcd3-1dc2f39df6ec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:37.088364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4bb6a-0104-40c5-bcd3-1dc2f39df6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:37.110859 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-40a4bb6a-0104-40c5-bcd3-1dc2f39df6ec None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:37.112062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-40a4bb6a-0104-40c5-bcd3-1dc2f39df6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:37.136958 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:37.200075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 73/160] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:37.579054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1aeb0c-6608-4716-b541-2046eeca0068 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:37.603708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ed27-348e-4602-8827-275caffbcce0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:37.604658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ed27-348e-4602-8827-275caffbcce0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:37.605123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566ed27-348e-4602-8827-275caffbcce0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:37.606936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 74/161] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:37.612168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7aa83-e3c8-436b-a4a3-b961c8614c70 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:37.612716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7aa83-e3c8-436b-a4a3-b961c8614c70 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:37.613242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7aa83-e3c8-436b-a4a3-b961c8614c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:37.637085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bca7aa83-e3c8-436b-a4a3-b961c8614c70 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:37.640462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bca7aa83-e3c8-436b-a4a3-b961c8614c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:37.797098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 75/162] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:37.800219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1aeb0c-6608-4716-b541-2046eeca0068 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:37.838077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7aa83-e3c8-436b-a4a3-b961c8614c70 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:37.873795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:37.874863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:37.877151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:37.877628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:37.900134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:37.910946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:37.912123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:37.913159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:37.915737 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 88/163] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:37.921424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:37.974512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:38.046951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:38.050060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7aa83-e3c8-436b-a4a3-b961c8614c70 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.092490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831e419-41f6-418a-a706-a0769cbe7732 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.106213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:38.106849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:38.107129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:38.107402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:38.122243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:38.131910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:38.132868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:38.133642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 76/164] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:58:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:38.150352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:38.170069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:38.170690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:38.171531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:38.171922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:38.172929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:38.173348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:38.174766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:38.192832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 89/165] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:38.198056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:38.301557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1cd29-36b0-4d71-ba7e-de9234e5dc56 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.342579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:38.349856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d3872-6066-4c2c-b8f3-f41992a7ff6e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.351076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:38.351759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:38.351953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:38.352222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:38.352878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:38.353174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:38.354576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:38.372206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 77/166] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:58:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:38.374215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:38.390725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:38.411217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:38.412204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:38.412693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:38.413161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] PATH_INFO: `/v3/users/9242a7b2cdf946a38262cb2b38f5cb44` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:38.414705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] RBAC: Authorizing `identity:get_user(user_id=9242a7b2cdf946a38262cb2b38f5cb44)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:38.416994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:38.424378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 90/167] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:38 2022] GET /identity/v3/users/9242a7b2cdf946a38262cb2b38f5cb44 => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:38.425261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:38.606403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d66695-8baa-452a-b990-a3af638b2080 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.632342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e150e25-8ecb-4a7e-ad85-16013aa485be None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.693227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:38.693915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:38.694199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:38.694473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:38.698440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:38.699913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:38.701538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 91/168] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:38 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:38.707117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:38.731776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:38.732871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:38.733557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:38.734115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:38.755617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:38.755826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:38.761809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:38.762724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:38.763538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:38.771520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 78/169] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:38.772600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:38.886189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:38.890587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c7395-e1ac-4b2a-817b-df3d4d8cab4e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.947520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa4b01-74bf-4f2f-b02e-283e71b67619 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:38.958632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:38.959374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:38.959774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:38.960065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.009915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36acc3-6f8a-4c30-b82c-70e9de1fc2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:39.010537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36acc3-6f8a-4c30-b82c-70e9de1fc2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.010821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36acc3-6f8a-4c30-b82c-70e9de1fc2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.011097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36acc3-6f8a-4c30-b82c-70e9de1fc2c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.093391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:39.095072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:39.152381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36acc3-6f8a-4c30-b82c-70e9de1fc2c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:39.153815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36acc3-6f8a-4c30-b82c-70e9de1fc2c1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:39.162084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:39.248979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 92/170] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:39.250624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:39.258876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:39.298894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 79/171] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:39.300245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb1e5-49d3-4b58-b997-29188349ffe0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.300539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb1e5-49d3-4b58-b997-29188349ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.300812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffb1e5-49d3-4b58-b997-29188349ffe0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.302163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 80/172] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:39.306322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d539e-2c49-4dea-a51b-d1a5ad558d02 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.306322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d539e-2c49-4dea-a51b-d1a5ad558d02 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.306322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d539e-2c49-4dea-a51b-d1a5ad558d02 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.306322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 81/173] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:39.306709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c09d1-4123-48e1-a1ca-1ebcbd89a3bf None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.307024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c09d1-4123-48e1-a1ca-1ebcbd89a3bf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.307220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c09d1-4123-48e1-a1ca-1ebcbd89a3bf None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.308563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 82/174] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:39.314655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c09d1-4123-48e1-a1ca-1ebcbd89a3bf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:39.364578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f2a0-2aa3-4c3d-874f-0203792547ff None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:39.397466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:39.404728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:39.406248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.406560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.407088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.407758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:39.408058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:39.409421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:39.418144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 93/175] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:39.418743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:39.428422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c09d1-4123-48e1-a1ca-1ebcbd89a3bf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:39.465255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d76183-19b7-43d2-80b3-c7ab16eb3fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:39.465924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d76183-19b7-43d2-80b3-c7ab16eb3fcc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.466416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d76183-19b7-43d2-80b3-c7ab16eb3fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.466522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d76183-19b7-43d2-80b3-c7ab16eb3fcc None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.468358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d76183-19b7-43d2-80b3-c7ab16eb3fcc None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:39.469985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d76183-19b7-43d2-80b3-c7ab16eb3fcc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:39.471786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 83/176] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:39 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:39.477474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394bc58-3918-473d-9856-80016b40a648 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.477951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394bc58-3918-473d-9856-80016b40a648 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.478578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394bc58-3918-473d-9856-80016b40a648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.496954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d394bc58-3918-473d-9856-80016b40a648 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:39.497791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d394bc58-3918-473d-9856-80016b40a648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:39.531921 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:39.538036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6fb0c-ec51-45a7-8d88-9b8f4f4a8a0c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:39.577872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 84/177] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:39.579709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf106f2-70a0-4efe-992f-85e1ae080ef7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.579999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf106f2-70a0-4efe-992f-85e1ae080ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.580272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf106f2-70a0-4efe-992f-85e1ae080ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.588133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46235a34-7c4a-4e63-b357-0ddcd3afbd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:39.589033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46235a34-7c4a-4e63-b357-0ddcd3afbd0a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.589566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46235a34-7c4a-4e63-b357-0ddcd3afbd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.590081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46235a34-7c4a-4e63-b357-0ddcd3afbd0a None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.591414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46235a34-7c4a-4e63-b357-0ddcd3afbd0a None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:39.594319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46235a34-7c4a-4e63-b357-0ddcd3afbd0a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:39.596515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 94/178] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:39 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:39.597873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c5365-d1da-4450-bcaa-e9ff49e1c0b5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.598387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c5365-d1da-4450-bcaa-e9ff49e1c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.598853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c5365-d1da-4450-bcaa-e9ff49e1c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.604264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cbf106f2-70a0-4efe-992f-85e1ae080ef7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:39.605458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cbf106f2-70a0-4efe-992f-85e1ae080ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:39.621473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b49c5365-d1da-4450-bcaa-e9ff49e1c0b5 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:39.623513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b49c5365-d1da-4450-bcaa-e9ff49e1c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:39.681846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 85/179] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:39.681846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf106f2-70a0-4efe-992f-85e1ae080ef7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:39.696105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:39.707078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 95/180] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:39.707735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c5365-d1da-4450-bcaa-e9ff49e1c0b5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:39.816150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c5365-d1da-4450-bcaa-e9ff49e1c0b5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:39.851771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:39.852381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.852663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.852935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.853856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=9242a7b2cdf946a38262cb2b38f5cb44) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:39.854169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:39.855696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:39.862095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 96/181] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:39 2022] GET /identity/v3/role_assignments?user.id=9242a7b2cdf946a38262cb2b38f5cb44&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:39.863257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:39.868188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:39.875932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf106f2-70a0-4efe-992f-85e1ae080ef7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:39.913051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36ac6c-8090-4fc8-83f5-60005cbab8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:39.913694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36ac6c-8090-4fc8-83f5-60005cbab8be None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.914001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36ac6c-8090-4fc8-83f5-60005cbab8be None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.914274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36ac6c-8090-4fc8-83f5-60005cbab8be None admin] PATH_INFO: `/v3/users/dda66a8495034a4f94e940f3175970f3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.915423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36ac6c-8090-4fc8-83f5-60005cbab8be None admin] RBAC: Authorizing `identity:get_user(user_id=dda66a8495034a4f94e940f3175970f3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:39.916872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36ac6c-8090-4fc8-83f5-60005cbab8be None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:39.918522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 86/182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:39 2022] GET /identity/v3/users/dda66a8495034a4f94e940f3175970f3 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:39.919697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4841df5-a7f7-4a2d-bf41-11f3fbf60f6d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:39.919976 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4841df5-a7f7-4a2d-bf41-11f3fbf60f6d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:39.920249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4841df5-a7f7-4a2d-bf41-11f3fbf60f6d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:39.922286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 87/183] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:39.922850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4841df5-a7f7-4a2d-bf41-11f3fbf60f6d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:39.932261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:39.985912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf2acd-1c2a-4b3e-8f7f-d7ddf90a400f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.028125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.028755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.029042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.029317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.030096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] RBAC: Adding query filter params (role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=dda66a8495034a4f94e940f3175970f3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:40.030610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:40.032186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:40.039583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 97/184] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:39 2022] GET /identity/v3/role_assignments?user.id=dda66a8495034a4f94e940f3175970f3&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:40.040268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.057806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4841df5-a7f7-4a2d-bf41-11f3fbf60f6d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.115850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53952b28-5dfb-4acd-a650-238ed94f9e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.116892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53952b28-5dfb-4acd-a650-238ed94f9e21 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.117484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53952b28-5dfb-4acd-a650-238ed94f9e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.117904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53952b28-5dfb-4acd-a650-238ed94f9e21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.131350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53952b28-5dfb-4acd-a650-238ed94f9e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-53952b28-5dfb-4acd-a650-238ed94f9e21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:40.137673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:40.138667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:40.139472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 88/185] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:40.145996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca37e3e-6122-4eb5-9528-9bcb53e7dde7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.145996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca37e3e-6122-4eb5-9528-9bcb53e7dde7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.145996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca37e3e-6122-4eb5-9528-9bcb53e7dde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.167204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3ca37e3e-6122-4eb5-9528-9bcb53e7dde7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:40.169065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3ca37e3e-6122-4eb5-9528-9bcb53e7dde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:40.183320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a012-4f3c-498d-b437-47ef941c44ef None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.193370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.216890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.242611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.243645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.244019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.244360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.266659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 89/186] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:40.267466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca37e3e-6122-4eb5-9528-9bcb53e7dde7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.330799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:40.332282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:40.398940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca37e3e-6122-4eb5-9528-9bcb53e7dde7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.440687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.441246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.441524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.441822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.451963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:40.458225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 98/187] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:40.458225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:40.458556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:40.459591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:40.460762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 90/188] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:40.464488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.479258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.484395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.569875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a70e13-4220-454e-85df-2b7f4e3763a0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.573914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a767fb-f8e8-41da-8d79-0bbd9cc2c6f2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.603246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.603866 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.604142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.604409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.605497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:40.606632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.606980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:40.607234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.607514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.607788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.609003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:40.609124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 99/189] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:40.609304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:40.610680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.610778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:40.620170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 91/190] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:40.620974 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.692984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.698381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.711601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add49d2-2d59-4125-a8f1-83a369313d21 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.718395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a4eb7-cf09-43cf-ad61-17c0ccf54c5d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.745456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.746112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.746387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.746660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] PATH_INFO: `/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.747770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] RBAC: Authorizing `identity:get_project(project_id=d7ec4926fdb8498bac0fa968d08a6b3d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:40.749341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:40.751015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 100/191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/projects/d7ec4926fdb8498bac0fa968d08a6b3d => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:40.751562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.751772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.752169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.752446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.752722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.753387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:40.753688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:40.756070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:40.764146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 92/192] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:40.765388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.851503 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eebef-7df2-4b0a-93cc-7a3f0962dd6a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.861154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d3468-b9a9-469f-9141-df751a7568a7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:40.884696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.885310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.885587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.885886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.893630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:40.894256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:40.894537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:40.894813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:40.895458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:40.895553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:40.896892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:40.900369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:40.901683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:40.902743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 101/193] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:40.904321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:40.906965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.920415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 93/194] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:58:40 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:40.925185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:40.927332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.023815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1e106-a606-4ee2-8425-dcda6d219966 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.038297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0dd09-74fb-4619-921c-39e5de95c52c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.068960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.069622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.069932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.070211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] PATH_INFO: `/v3/users/9a6759087cc445818255df5e5b053bc5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.083904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.084522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.084800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.085072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.085827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=d7ec4926fdb8498bac0fa968d08a6b3d, user.id=dda66a8495034a4f94e940f3175970f3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:41.086138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.087703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.087965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] RBAC: Authorizing `identity:get_user(user_id=9a6759087cc445818255df5e5b053bc5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.089867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.095732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 102/195] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/users/9a6759087cc445818255df5e5b053bc5 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:41.095732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.099811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 94/196] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:40 2022] GET /identity/v3/role_assignments?user.id=dda66a8495034a4f94e940f3175970f3&scope.project.id=d7ec4926fdb8498bac0fa968d08a6b3d&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:41.110707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.198516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:41.201560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:41.236160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bbd15f-3e8c-46c6-8d70-24e9a9b7c33e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.263744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551892c-9a76-4bbb-a372-548336b312bf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.281960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.282532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.282829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.283080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] PATH_INFO: `/v3/users/9a6759087cc445818255df5e5b053bc5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.284247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6759087cc445818255df5e5b053bc5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.285706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.288322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 103/197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/users/9a6759087cc445818255df5e5b053bc5 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:41.289127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.313965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.315188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.315672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.316170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.317307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:41.317618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.319521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.331815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 95/198] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:41.332819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.429992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2ef32-29b2-4388-93a6-96ef5eff0593 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.474785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fecc0-7586-4e52-bc72-bf7a969a594f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.475490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.476355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.476696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.477928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.495041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:41.498002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:41.521054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.526448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.526743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.526991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.571576 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.573789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.618694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.620319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.676637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 104/199] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:41.678162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.724090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:41.741608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 96/200] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:41.742877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.768086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:41.793168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552b3b4-bc51-4d03-83cf-3e6034c82c27 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.843995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.844369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.844651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.845548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] PATH_INFO: `/v3/users/b52a0d8269c24af3ad837563e11e4fdd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.847051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] RBAC: Authorizing `identity:get_user(user_id=b52a0d8269c24af3ad837563e11e4fdd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.849180 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.852517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 105/201] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/users/b52a0d8269c24af3ad837563e11e4fdd => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:41.853384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.871711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d493aeb-dde9-4257-a628-112eac0a6e09 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.908585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:41.909589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:41.910089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:41.910577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:41.911927 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:41.913667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:41.915854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 97/202] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:41.916895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:41.972739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488a29d-4f1d-4db2-af6c-4f415cdeb6ce None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:41.987142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:42.008440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:42.018183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:42.018817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:42.019098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:42.019369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:42.021009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:42.022757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:42.024392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 106/203] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:42.025232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:42.027598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb809b-d061-463f-9eb3-0a0ddb047cac None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:42.064462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:42.065488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:42.066035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:42.066529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:42.147064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbe09e-f2f2-48f4-a8b3-93fcdce82690 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:42.157054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:42.159116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:42.188048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:42.188953 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:42.189408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:42.189917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:42.190790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=9a6759087cc445818255df5e5b053bc5) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:42.191090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:42.192756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:42.199360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 107/204] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:42 2022] GET /identity/v3/role_assignments?user.id=9a6759087cc445818255df5e5b053bc5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:42.200341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:42.259913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:42.262593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:42.281816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 98/205] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:42.287706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:42.333006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01d10e-868d-4aa4-aff6-d9320e3ee0e0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:42.373577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:42.374868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:42.375300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:42.375743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:42.376750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] RBAC: Adding query filter params (role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=9a6759087cc445818255df5e5b053bc5) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:42.377208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:42.378971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:42.386903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 108/206] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:42 2022] GET /identity/v3/role_assignments?user.id=9a6759087cc445818255df5e5b053bc5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:42.446109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dfffd-cea5-4f64-9857-8169bb59a781 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:42.485408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe0350-87e5-4352-b4c4-6e10ec02dde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:42.486108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe0350-87e5-4352-b4c4-6e10ec02dde8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:42.486370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe0350-87e5-4352-b4c4-6e10ec02dde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:42.486641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe0350-87e5-4352-b4c4-6e10ec02dde8 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:42.487792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe0350-87e5-4352-b4c4-6e10ec02dde8 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:42.489228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe0350-87e5-4352-b4c4-6e10ec02dde8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:42.491191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 99/207] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:42 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:42.497263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:42.630645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b93989-75f4-4140-900c-6da7669ede04 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:42.650111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:42.668787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6781b-d28c-4476-8b78-fd5100c88320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:42.669400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6781b-d28c-4476-8b78-fd5100c88320 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:42.669678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6781b-d28c-4476-8b78-fd5100c88320 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:42.669977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6781b-d28c-4476-8b78-fd5100c88320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:42.670706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6781b-d28c-4476-8b78-fd5100c88320 None admin] RBAC: Adding query filter params (role.id=969ed0ca7dc147ecad313e6a3909656d, scope.project.id=18b7b138b42148d79097c7b2e243b955, user.id=b52a0d8269c24af3ad837563e11e4fdd) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:42.671170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6781b-d28c-4476-8b78-fd5100c88320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:42.672688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6781b-d28c-4476-8b78-fd5100c88320 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:42.679278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 109/208] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:42 2022] GET /identity/v3/role_assignments?user.id=b52a0d8269c24af3ad837563e11e4fdd&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=969ed0ca7dc147ecad313e6a3909656d => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:43.499787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3449-9367-490b-b109-df4fd2adb482 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:43.500293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3449-9367-490b-b109-df4fd2adb482 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:43.500450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3449-9367-490b-b109-df4fd2adb482 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:43.502416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 100/209] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:43.509160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a7074-3258-4c92-9557-571de1e4f7c4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:43.509617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a7074-3258-4c92-9557-571de1e4f7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:43.510318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a7074-3258-4c92-9557-571de1e4f7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:43.549780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d03a7074-3258-4c92-9557-571de1e4f7c4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:43.550983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d03a7074-3258-4c92-9557-571de1e4f7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:43.626739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 110/210] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:43.681565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1049dc-4d23-4f1e-9236-7d4187d8d7fd None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:43.681962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1049dc-4d23-4f1e-9236-7d4187d8d7fd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:43.682314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1049dc-4d23-4f1e-9236-7d4187d8d7fd None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:43.683889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 101/211] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:43.689470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a96bd4-77b9-4f40-bd76-c6dd99338d69 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:43.689790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a96bd4-77b9-4f40-bd76-c6dd99338d69 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:43.690075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a96bd4-77b9-4f40-bd76-c6dd99338d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:43.708654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-76a96bd4-77b9-4f40-bd76-c6dd99338d69 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:43.710109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-76a96bd4-77b9-4f40-bd76-c6dd99338d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:43.788208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 111/212] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:43.918029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1049dc-4d23-4f1e-9236-7d4187d8d7fd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:43.977129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:44.023224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1049dc-4d23-4f1e-9236-7d4187d8d7fd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:44.059831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a96bd4-77b9-4f40-bd76-c6dd99338d69 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:44.063170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:44.064847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.065325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.065820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.079749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:44.085713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:44.086610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:44.087391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:44.087982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 102/213] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:44.094961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:44.100287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:44.182353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a96bd4-77b9-4f40-bd76-c6dd99338d69 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:44.221497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:44.222172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.222447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.222889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.224016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:44.225448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:44.227238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 112/214] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:44.232535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:44.239065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e967ad2-e42b-483d-b1ab-c9f3170c2807 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:44.277121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:44.277752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.278048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.278320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.278991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:44.279289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:44.280844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:44.283642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:44.295619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 103/215] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:44.297337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:44.347511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:44.352780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef1327-51f0-4b1d-a7a9-33d5cd4f53a3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:44.389049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:44.390220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.390502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.390769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.391569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:44.393082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:44.424793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8732f8b-ec06-4ee2-8844-3fffde6d903a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:44.426327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 113/216] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:58:44 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:44.465313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c13c2-352f-40f5-95f5-40a9bfd828d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:44.466348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c13c2-352f-40f5-95f5-40a9bfd828d7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.466851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c13c2-352f-40f5-95f5-40a9bfd828d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.467315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c13c2-352f-40f5-95f5-40a9bfd828d7 None admin] PATH_INFO: `/v3/users/2d916167c863408aa7c93c578c109e9d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.484702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c13c2-352f-40f5-95f5-40a9bfd828d7 None admin] RBAC: Authorizing `identity:get_user(user_id=2d916167c863408aa7c93c578c109e9d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:44.486587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c13c2-352f-40f5-95f5-40a9bfd828d7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:44.488587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 104/217] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:44 2022] GET /identity/v3/users/2d916167c863408aa7c93c578c109e9d => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:44.494541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:44.630419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d41092-71f4-48cd-9cc6-17dbabf50143 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:44.665160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0865c8-008b-43f3-b423-b3bcf556fcaa None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.666011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0865c8-008b-43f3-b423-b3bcf556fcaa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.666281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0865c8-008b-43f3-b423-b3bcf556fcaa None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.667735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 105/218] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:44.668171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:44.668794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.669131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.669408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.677014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8aaf-d8e7-4c4e-beb4-3b510e921ae0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.677320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8aaf-d8e7-4c4e-beb4-3b510e921ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.677553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c8aaf-d8e7-4c4e-beb4-3b510e921ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.698516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:44.703988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db6c8aaf-d8e7-4c4e-beb4-3b510e921ae0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:44.705373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db6c8aaf-d8e7-4c4e-beb4-3b510e921ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:44.743818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:44.751465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:44.752938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:44.806223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 106/219] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:44.811197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99276-94a1-47a3-9958-4551555bcfee None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.811480 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99276-94a1-47a3-9958-4551555bcfee None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.812318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99276-94a1-47a3-9958-4551555bcfee None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.813633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 107/220] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:44.819950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfa520-e876-4b16-ab6e-1a3a2539231b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:44.820227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfa520-e876-4b16-ab6e-1a3a2539231b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:44.820493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfa520-e876-4b16-ab6e-1a3a2539231b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:44.845362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d9cfa520-e876-4b16-ab6e-1a3a2539231b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:44.846589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9cfa520-e876-4b16-ab6e-1a3a2539231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:44.878519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 114/221] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:44.884272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:44.933635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 108/222] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:44.957710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:45.005949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee5b34-05a4-4e62-b698-c1ff16da7902 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:45.042950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:45.043598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:45.043879 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:45.044154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:45.045307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:45.047737 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:45.049695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 115/223] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:44 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:45.055289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfa520-e876-4b16-ab6e-1a3a2539231b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:45.136615 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:45.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:45.186136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfa520-e876-4b16-ab6e-1a3a2539231b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:45.230893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:45.231518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:45.231802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:45.232078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:45.232823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=2d916167c863408aa7c93c578c109e9d) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:45.233201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:45.236872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:45.241343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 109/224] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/role_assignments?user.id=2d916167c863408aa7c93c578c109e9d&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:45.274724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:45.364078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729394a-39fe-4260-829d-6d9522ce4486 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:45.434156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:45.440177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:45.441158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:45.441392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:45.442122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:45.453427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:45.458748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:45.459647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:45.460449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 116/225] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:45.465704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:45.471238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c32851-712f-43c1-ae6c-a034db59175a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:45.508986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:45.509628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:45.509938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:45.510214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:45.527924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:45.535369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:45.536276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:45.537063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 110/226] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:45.548994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:45.578146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:45.688892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b2060-36f5-41bd-9a6a-78a278074cca None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:45.730395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:45.731012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:45.731293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:45.731571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:45.731854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefac47-f616-4353-8ffd-000c57c1dd7e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:45.732282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:45.732543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:45.735001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:45.744628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 117/227] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:45.750437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:45.804194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:45.834448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:45.835079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:45.835364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:45.835642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:45.836322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:45.836632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:45.838055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:45.855306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 111/228] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:45.862150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:45.864633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa19c53-d3d1-447d-9126-9d52bc135b58 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:45.904361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:45.904999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:45.905399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:45.906480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] PATH_INFO: `/v3/users/9a6759087cc445818255df5e5b053bc5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:45.907956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6759087cc445818255df5e5b053bc5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:45.909431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:45.911496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 118/229] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/users/9a6759087cc445818255df5e5b053bc5 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:45.917026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:46.031104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d03cb-e602-4bf0-8be7-dc464d5ccec6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:46.046692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:46.069382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:46.070073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:46.070402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:46.070684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:46.071839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:46.073281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:46.075313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 119/230] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:46.081892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:46.114530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:46.154136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5335a5b-594f-4cd8-84f9-66fc49bbb4f5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:46.209230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68406-6c56-4fc7-8c4d-7551bf6ec673 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:46.248516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:46.250489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:46.251075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:46.251577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/9a6759087cc445818255df5e5b053bc5/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:46.253908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] RBAC: Authorizing `identity:create_grant(project_id=883fcb9f0c0c404586e1b021177d762e, user_id=9a6759087cc445818255df5e5b053bc5, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:46.255566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:46.258437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:46.259358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:46.259488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:46.259650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] PATH_INFO: `/v3/users/9a6759087cc445818255df5e5b053bc5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:46.260981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6759087cc445818255df5e5b053bc5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:46.264222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:46.264222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 112/231] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:45 2022] GET /identity/v3/users/9a6759087cc445818255df5e5b053bc5 => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:46.270966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:46.278336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 120/232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:46 2022] PUT /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/9a6759087cc445818255df5e5b053bc5/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:58:46.445911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473b436-fd36-46be-ada7-241b3da3ac55 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:46.484844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a1952-bb5d-492f-8917-d8bf1d39c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:46.485485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1952-bb5d-492f-8917-d8bf1d39c750 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:46.485793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1952-bb5d-492f-8917-d8bf1d39c750 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:46.486076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a1952-bb5d-492f-8917-d8bf1d39c750 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:46.487227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a1952-bb5d-492f-8917-d8bf1d39c750 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:46.488678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a1952-bb5d-492f-8917-d8bf1d39c750 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:46.490354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 113/233] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:46 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:46.496346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:46.543704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:46.607936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77b505-6a61-477a-86e2-8f746f18d350 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:46.647728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de65d9-90b1-437d-b26c-129c69809fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:46.648408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de65d9-90b1-437d-b26c-129c69809fbc None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:46.648754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de65d9-90b1-437d-b26c-129c69809fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:46.649035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de65d9-90b1-437d-b26c-129c69809fbc None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/9a6759087cc445818255df5e5b053bc5/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:46.652788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de65d9-90b1-437d-b26c-129c69809fbc None admin] RBAC: Authorizing `identity:create_grant(project_id=883fcb9f0c0c404586e1b021177d762e, user_id=9a6759087cc445818255df5e5b053bc5, role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:46.654292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de65d9-90b1-437d-b26c-129c69809fbc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:46.677717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 121/234] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:46 2022] PUT /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/9a6759087cc445818255df5e5b053bc5/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:47.407478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98983-274d-455c-8b80-132a7decf1ae None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:47.407992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98983-274d-455c-8b80-132a7decf1ae None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:47.408067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98983-274d-455c-8b80-132a7decf1ae None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:47.409651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 114/235] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:47.414832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bacc3-a9b6-45d8-85e4-57bf10988c6a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:47.415236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bacc3-a9b6-45d8-85e4-57bf10988c6a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:47.415404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bacc3-a9b6-45d8-85e4-57bf10988c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:47.442951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-728bacc3-a9b6-45d8-85e4-57bf10988c6a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:47.444291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-728bacc3-a9b6-45d8-85e4-57bf10988c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:47.540064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3ca3b-2673-4bf3-9a2d-de13f8811b0a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:47.540340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3ca3b-2673-4bf3-9a2d-de13f8811b0a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:47.540622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3ca3b-2673-4bf3-9a2d-de13f8811b0a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:47.541997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 115/236] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:47.551713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7258c-5d0a-4da3-afa5-260e121c5a9f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:47.551989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7258c-5d0a-4da3-afa5-260e121c5a9f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:47.552266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7258c-5d0a-4da3-afa5-260e121c5a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:47.565112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 122/237] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:47.573787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-02c7258c-5d0a-4da3-afa5-260e121c5a9f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:47.575034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-02c7258c-5d0a-4da3-afa5-260e121c5a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:47.588604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:47.662167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 116/238] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:47.878709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bacc3-a9b6-45d8-85e4-57bf10988c6a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:47.914703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:47.966871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7258c-5d0a-4da3-afa5-260e121c5a9f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:48.013435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bacc3-a9b6-45d8-85e4-57bf10988c6a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:48.056785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2fa48-1c5c-4279-9644-477eb3c7766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:48.058161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2fa48-1c5c-4279-9644-477eb3c7766f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.058885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2fa48-1c5c-4279-9644-477eb3c7766f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.059302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2fa48-1c5c-4279-9644-477eb3c7766f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.060707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2fa48-1c5c-4279-9644-477eb3c7766f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:48.062748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2fa48-1c5c-4279-9644-477eb3c7766f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:48.069117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 123/239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:48.069117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db96dc0-73e9-475e-8ba4-5eaef656e76b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.069117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db96dc0-73e9-475e-8ba4-5eaef656e76b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.069117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db96dc0-73e9-475e-8ba4-5eaef656e76b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.071897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 124/240] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:48.071897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db96dc0-73e9-475e-8ba4-5eaef656e76b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:48.086842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7258c-5d0a-4da3-afa5-260e121c5a9f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:48.143652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ba2bb-67d4-4ad3-83db-0019b5428207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:48.144304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba2bb-67d4-4ad3-83db-0019b5428207 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.144586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba2bb-67d4-4ad3-83db-0019b5428207 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.144858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba2bb-67d4-4ad3-83db-0019b5428207 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.160492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ba2bb-67d4-4ad3-83db-0019b5428207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-852ba2bb-67d4-4ad3-83db-0019b5428207 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:48.166723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:48.167589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:48.168363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 117/241] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:48.174911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af8da9-7226-45ef-bfbb-45151aefd561 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.174911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af8da9-7226-45ef-bfbb-45151aefd561 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.174911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af8da9-7226-45ef-bfbb-45151aefd561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.185669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:48.194647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b2af8da9-7226-45ef-bfbb-45151aefd561 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:48.196101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b2af8da9-7226-45ef-bfbb-45151aefd561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:48.210689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:48.218445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db96dc0-73e9-475e-8ba4-5eaef656e76b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:48.262634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:48.263626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.264129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.264609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.266072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:48.267765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:48.297889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 118/243] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:48.298243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 125/242] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:58:48 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:48.298611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af8da9-7226-45ef-bfbb-45151aefd561 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:48.412681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af8da9-7226-45ef-bfbb-45151aefd561 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:48.450538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:48.451178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.451466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.451748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.452648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:48.453266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:48.454679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:48.457668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:48.465799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 119/244] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:48.475829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:48.655008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f099361-2f18-4be2-aa34-ffcc1ea42d6b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:48.690768 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:48.691140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:48.691413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.691696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.691967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] PATH_INFO: `/v3/users/81810b1993724d9084304b2ae45da381` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.706890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] RBAC: Authorizing `identity:get_user(user_id=81810b1993724d9084304b2ae45da381)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:48.708640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:48.710700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 126/245] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:48 2022] GET /identity/v3/users/81810b1993724d9084304b2ae45da381 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:48.719271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:48.737978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:48.824197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01f190-e644-4d5a-a3c1-e74f800490bd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:48.861874 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546ae7d-760a-456f-b2f6-956e80b6161f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:48.869506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2fb7c-9884-4855-ac8c-2e04de731470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:48.870451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2fb7c-9884-4855-ac8c-2e04de731470 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.870924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2fb7c-9884-4855-ac8c-2e04de731470 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.871387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2fb7c-9884-4855-ac8c-2e04de731470 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.886160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2fb7c-9884-4855-ac8c-2e04de731470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f7b2fb7c-9884-4855-ac8c-2e04de731470 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:48.892011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:48.893121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:48.894154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:48.896384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 120/246] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:48.897839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf36867-a163-4bdd-9884-42cdd23947f0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.898293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf36867-a163-4bdd-9884-42cdd23947f0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.898567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf36867-a163-4bdd-9884-42cdd23947f0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.900024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 121/247] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:48.903683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf36867-a163-4bdd-9884-42cdd23947f0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:48.904271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:48.905072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:48.905559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:48.906271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:48.991382 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:49.043474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf36867-a163-4bdd-9884-42cdd23947f0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:49.044435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:49.046689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:49.084027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd796b7-2dc4-421a-a65d-43ce1adcb794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:49.084637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd796b7-2dc4-421a-a65d-43ce1adcb794 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:49.084912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd796b7-2dc4-421a-a65d-43ce1adcb794 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:49.085181 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd796b7-2dc4-421a-a65d-43ce1adcb794 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:49.085877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd796b7-2dc4-421a-a65d-43ce1adcb794 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:49.086178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd796b7-2dc4-421a-a65d-43ce1adcb794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:49.089750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd796b7-2dc4-421a-a65d-43ce1adcb794 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:49.089889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:49.105682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 122/248] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:49.110191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05521ab4-dba4-444b-b818-2972387ecbf5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:49.110330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05521ab4-dba4-444b-b818-2972387ecbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:49.111618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05521ab4-dba4-444b-b818-2972387ecbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:49.137759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-05521ab4-dba4-444b-b818-2972387ecbf5 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:49.137759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05521ab4-dba4-444b-b818-2972387ecbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:49.241832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 123/249] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:49.241832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05521ab4-dba4-444b-b818-2972387ecbf5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:49.348776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 127/250] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:49.361758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:49.405797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:49.546811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05521ab4-dba4-444b-b818-2972387ecbf5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:49.607590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5cacd-8bf7-412e-9ea9-8c2718290b76 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:49.629609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:49.630629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:49.631101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:49.631606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] PATH_INFO: `/v3/users/2d916167c863408aa7c93c578c109e9d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:49.634873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] RBAC: Authorizing `identity:get_user(user_id=2d916167c863408aa7c93c578c109e9d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:49.636864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:49.647969 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 124/251] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:49 2022] GET /identity/v3/users/2d916167c863408aa7c93c578c109e9d => generated 297 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:49.649309 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:49.656707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:49.730936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:49.730936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:49.730936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:49.730936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:49.731337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:49.731767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:49.734144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 128/252] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:49 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:49.735959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:49.837054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a9ba0-2fa6-488a-aeba-cb19cda10b50 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:49.855739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:49.874979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:49.875887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:49.876169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:49.876446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:49.887258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:49.891407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd85c9-d07f-4d1c-b9d4-39ab2808b4eb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:49.893316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:49.894235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:58:49.895022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:49.899327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 125/253] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:49.900348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:49.939994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c06f2d-57f9-4e79-927a-6c16b5cbe42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:49.940968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06f2d-57f9-4e79-927a-6c16b5cbe42d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:49.941428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06f2d-57f9-4e79-927a-6c16b5cbe42d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:49.941901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c06f2d-57f9-4e79-927a-6c16b5cbe42d None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:49.943246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06f2d-57f9-4e79-927a-6c16b5cbe42d None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:49.945384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c06f2d-57f9-4e79-927a-6c16b5cbe42d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:49.948145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 129/254] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:49 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:49.949834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3c50-8127-4f19-976d-4509b8e23352 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:49.950304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3c50-8127-4f19-976d-4509b8e23352 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:49.951047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c3c50-8127-4f19-976d-4509b8e23352 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:49.953154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 130/255] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:49.954209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c3c50-8127-4f19-976d-4509b8e23352 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:50.014804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3aa0-8548-4e81-b969-ee0f0c6adcac None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:50.028033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:50.051907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:50.052514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:50.052795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:50.053067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:50.053829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] RBAC: Adding query filter params (role.id=969ed0ca7dc147ecad313e6a3909656d, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=81810b1993724d9084304b2ae45da381) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:50.054140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:50.055587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:50.062324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 126/256] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:49 2022] GET /identity/v3/role_assignments?user.id=81810b1993724d9084304b2ae45da381&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=969ed0ca7dc147ecad313e6a3909656d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:50.063348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:50.101044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c3c50-8127-4f19-976d-4509b8e23352 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:50.119686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:50.184571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56039704-186a-448b-b1f9-5e5b8b019bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:50.185201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56039704-186a-448b-b1f9-5e5b8b019bee None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:50.185484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56039704-186a-448b-b1f9-5e5b8b019bee None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:50.185778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56039704-186a-448b-b1f9-5e5b8b019bee None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:50.186470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56039704-186a-448b-b1f9-5e5b8b019bee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:50.187321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56039704-186a-448b-b1f9-5e5b8b019bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:50.189535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56039704-186a-448b-b1f9-5e5b8b019bee None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:50.207355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 131/257] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:50.208679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc653d9-c09d-480d-a14a-1e38d50c9db0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:50.208945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc653d9-c09d-480d-a14a-1e38d50c9db0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:50.209220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc653d9-c09d-480d-a14a-1e38d50c9db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:50.236789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2b806-0aca-42a1-b937-3e018646f60d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:50.257907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ffc653d9-c09d-480d-a14a-1e38d50c9db0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:50.259162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ffc653d9-c09d-480d-a14a-1e38d50c9db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:50.304775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:50.305509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:50.305815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:50.306093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/2d916167c863408aa7c93c578c109e9d/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:50.309326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=883fcb9f0c0c404586e1b021177d762e, user_id=2d916167c863408aa7c93c578c109e9d, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:50.311122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:50.341269 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 127/258] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:50 2022] PUT /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/2d916167c863408aa7c93c578c109e9d/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:50.342772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:50.402994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 132/259] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:50.497877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1916707-4979-4b50-b47b-f0ada8e8ebb7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:50.500434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:50.548299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:50.548977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:50.549221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:50.549496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] PATH_INFO: `/v3/users/9a6759087cc445818255df5e5b053bc5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:50.554574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] RBAC: Authorizing `identity:get_user(user_id=9a6759087cc445818255df5e5b053bc5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:50.556103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:50.558034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 128/260] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:50 2022] GET /identity/v3/users/9a6759087cc445818255df5e5b053bc5 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:50.564592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc653d9-c09d-480d-a14a-1e38d50c9db0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:50.631880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:50.698436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc653d9-c09d-480d-a14a-1e38d50c9db0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:50.739124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:50.739754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:50.740038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:50.740307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:50.817653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:50.819177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:50.862113 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:50.924538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 133/261] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:50.930196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:50.984519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:51.085944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacca21-28ed-46e7-b4da-93a47eb3acfb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:51.153198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:51.153890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:51.154173 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:51.154445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:51.158474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:51.160117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:51.162248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 129/262] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:50 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:51.167786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:51.175259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb1bdb-1f57-49bd-bf08-e2289df0c30a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:51.239853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:51.240420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:51.240696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:51.240965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:51.250222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:51.260338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:51.269227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:51.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:51.275078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:58:51.275865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:51.282082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 134/263] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:58:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:51.282973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:51.370626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a9b4-3fd6-4088-bd96-382dc9521374 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:51.419877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f86e0c-d510-4b40-aa4e-1cad7bf77ccd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:51.434441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:51.435057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:51.435339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:51.435620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:51.436353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=9a6759087cc445818255df5e5b053bc5) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:51.436694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:51.438186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:51.449891 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 130/264] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:51 2022] GET /identity/v3/role_assignments?user.id=9a6759087cc445818255df5e5b053bc5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:51.462827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:51.463897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:51.464442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:51.465073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:51.466054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:51.466583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:51.469268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:51.479156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 135/265] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:58:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:51.485167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:51.627717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d889f9-6bc8-4ae7-8498-15531db6bcd3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:51.629365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:51.665512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b12fa-23b9-44fc-8b8a-d7473f88ed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:51.666189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b12fa-23b9-44fc-8b8a-d7473f88ed05 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:51.666467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b12fa-23b9-44fc-8b8a-d7473f88ed05 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:51.666738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b12fa-23b9-44fc-8b8a-d7473f88ed05 None admin] PATH_INFO: `/v3/users/9a6759087cc445818255df5e5b053bc5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:51.667901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b12fa-23b9-44fc-8b8a-d7473f88ed05 None admin] RBAC: Authorizing `identity:get_user(user_id=9a6759087cc445818255df5e5b053bc5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:51.669353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b12fa-23b9-44fc-8b8a-d7473f88ed05 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:51.671923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 131/266] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:51 2022] GET /identity/v3/users/9a6759087cc445818255df5e5b053bc5 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:51.677177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:51.756518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:51.804953 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34efb5e-7623-4e6d-9516-0d8f47fe14a3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:51.848938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:51.849858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:51.850311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:51.850779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:51.941259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:51.943025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:52.023075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:52.028625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9d5cc-ef87-48c8-9a78-d3742c8ed0f8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.028854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9d5cc-ef87-48c8-9a78-d3742c8ed0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.029127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9d5cc-ef87-48c8-9a78-d3742c8ed0f8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.031186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 132/267] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:52.036637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae60c9-cdeb-447c-a66e-bb35871324cd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.036913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae60c9-cdeb-447c-a66e-bb35871324cd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.037185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae60c9-cdeb-447c-a66e-bb35871324cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.073156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 136/268] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:52.073750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-08ae60c9-cdeb-447c-a66e-bb35871324cd None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:52.074955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08ae60c9-cdeb-447c-a66e-bb35871324cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:52.083410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:52.181722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 133/269] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:52.236302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e696b15-d709-42eb-b036-e84b88054df7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:52.285759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd893f0-2f36-44b9-b034-2a3ec874aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:52.286682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd893f0-2f36-44b9-b034-2a3ec874aba3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.287135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd893f0-2f36-44b9-b034-2a3ec874aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.287583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd893f0-2f36-44b9-b034-2a3ec874aba3 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.288903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd893f0-2f36-44b9-b034-2a3ec874aba3 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:52.290533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd893f0-2f36-44b9-b034-2a3ec874aba3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:52.293588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 137/270] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:52 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:52.298811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae60c9-cdeb-447c-a66e-bb35871324cd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:52.353861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554f59-16dc-4659-a411-21a18b142ae3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.354056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554f59-16dc-4659-a411-21a18b142ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.354326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554f59-16dc-4659-a411-21a18b142ae3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.355783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 138/271] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:52.362290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ea3f9-9e04-4a60-af7d-888e86dba168 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.362777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ea3f9-9e04-4a60-af7d-888e86dba168 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.363125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ea3f9-9e04-4a60-af7d-888e86dba168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.384516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-449ea3f9-9e04-4a60-af7d-888e86dba168 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:52.386127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-449ea3f9-9e04-4a60-af7d-888e86dba168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:52.414990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:52.473878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 139/272] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:52.482830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae60c9-cdeb-447c-a66e-bb35871324cd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:52.484932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:52.527735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:52.527735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.527735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.527735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.528135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] RBAC: Adding query filter params (role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=9a6759087cc445818255df5e5b053bc5) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:52.528135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:52.529289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:52.536760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 134/273] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:52 2022] GET /identity/v3/role_assignments?user.id=9a6759087cc445818255df5e5b053bc5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:52.579324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ea3f9-9e04-4a60-af7d-888e86dba168 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:52.704205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ea3f9-9e04-4a60-af7d-888e86dba168 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:52.743121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80047547-fe61-4151-806a-1321006353a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:52.744145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047547-fe61-4151-806a-1321006353a4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.744621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047547-fe61-4151-806a-1321006353a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.745109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80047547-fe61-4151-806a-1321006353a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.756916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80047547-fe61-4151-806a-1321006353a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-80047547-fe61-4151-806a-1321006353a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:52.762158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:52.763059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:52.763875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:52.767042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 140/274] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:52.772951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:52.885187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93a468-58b3-4c99-b36d-a8b0998630e7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:52.897870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80047547-fe61-4151-806a-1321006353a4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:52.925810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:52.926430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:52.926714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:52.926988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:52.928048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:52.928048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:52.929404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:52.930068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:52.945386 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 135/275] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:52.948819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:52.980381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:53.027895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80047547-fe61-4151-806a-1321006353a4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:53.069109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b82e4b5-9e88-4704-a9b0-cc1ba2f2fc48 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:53.069329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:53.069836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.070122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.070398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.071543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:53.073001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:53.074918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 141/276] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:53.081081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:53.113237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:53.114683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.115654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.116409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] PATH_INFO: `/v3/users/03f962f0f47a4868aae769753ea79105` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.137384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=03f962f0f47a4868aae769753ea79105)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:53.140687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:53.144317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 136/277] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:52 2022] GET /identity/v3/users/03f962f0f47a4868aae769753ea79105 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:53.151015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:53.196246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:53.209985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7954449-c8bb-4ec8-9bd3-da07d426dc23 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:53.246891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbedbe-27ba-4914-b6b8-19af69d186f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:53.247932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbedbe-27ba-4914-b6b8-19af69d186f5 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.248434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbedbe-27ba-4914-b6b8-19af69d186f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.248951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbedbe-27ba-4914-b6b8-19af69d186f5 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.250449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbedbe-27ba-4914-b6b8-19af69d186f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:53.252179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbedbe-27ba-4914-b6b8-19af69d186f5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:53.265676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:53.278510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 142/278] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:58:53 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:53.279639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24d73f-9237-4613-ab9f-43772794da32 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.279929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24d73f-9237-4613-ab9f-43772794da32 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.280500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24d73f-9237-4613-ab9f-43772794da32 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.281675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 143/279] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:53.289101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23d480-396d-4d93-9d40-8c26a6956b6a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.289442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23d480-396d-4d93-9d40-8c26a6956b6a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.289648 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23d480-396d-4d93-9d40-8c26a6956b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.302473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2667a61-c30d-4899-94a8-f9ff321c1fe9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:53.320798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3c23d480-396d-4d93-9d40-8c26a6956b6a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:53.322397 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c23d480-396d-4d93-9d40-8c26a6956b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:53.343964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f372c04-4321-4ec7-99f0-535da630195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:53.344593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372c04-4321-4ec7-99f0-535da630195b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.344871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372c04-4321-4ec7-99f0-535da630195b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.345142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372c04-4321-4ec7-99f0-535da630195b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.345911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372c04-4321-4ec7-99f0-535da630195b None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, user.id=03f962f0f47a4868aae769753ea79105, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:53.346250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372c04-4321-4ec7-99f0-535da630195b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:53.347759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372c04-4321-4ec7-99f0-535da630195b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:53.354011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 137/280] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 13 13:58:53 2022] GET /identity/v3/role_assignments?user.id=03f962f0f47a4868aae769753ea79105&scope.system=all&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:53.371460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc5a0c-aea9-4975-8f4b-6a02e1a134e0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.371746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc5a0c-aea9-4975-8f4b-6a02e1a134e0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.372023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc5a0c-aea9-4975-8f4b-6a02e1a134e0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.373811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 138/281] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:53.379877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dd3e9-d21a-4d6b-9024-77bfafda4645 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.380114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dd3e9-d21a-4d6b-9024-77bfafda4645 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.380390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dd3e9-d21a-4d6b-9024-77bfafda4645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.403358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-509dd3e9-d21a-4d6b-9024-77bfafda4645 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:53.405689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-509dd3e9-d21a-4d6b-9024-77bfafda4645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:53.429132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 144/282] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:53.500903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 139/283] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:53.714114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23d480-396d-4d93-9d40-8c26a6956b6a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:53.772108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dd3e9-d21a-4d6b-9024-77bfafda4645 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:53.806550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:53.838944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23d480-396d-4d93-9d40-8c26a6956b6a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:53.902075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:53.902717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:53.905173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:53.907136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:53.924445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:53.931467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:53.932629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:53.933627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 145/284] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:53.938825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:53.939341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:53.984812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dd3e9-d21a-4d6b-9024-77bfafda4645 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.051580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.053278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.055913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.056338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:54.076511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:54.086808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:54.087723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:54.088539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 140/285] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:54.096156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:54.118010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0a770-2a7e-4114-8770-6bfd3c7ff8c3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.187487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.188516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.188813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.189095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:54.189797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:54.190111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:54.191486 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:54.193103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:54.211814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 146/286] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:54.218062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:54.250552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9d55b-a02b-4ccf-9ec6-c10524ef779e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.325773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.327295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.328625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.329043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:54.330563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:54.330976 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:54.332394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:54.342069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:54.349528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 141/287] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:54.355067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:54.379192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b028963-cf73-40c5-bfb2-19eac5e33e57 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.416408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.417426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.417947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.418414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] PATH_INFO: `/v3/users/81810b1993724d9084304b2ae45da381` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:54.419740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=81810b1993724d9084304b2ae45da381)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:54.421818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:54.423621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 147/288] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:54 2022] GET /identity/v3/users/81810b1993724d9084304b2ae45da381 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:54.429240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:54.504515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:54.566581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2905cd-a036-419b-b9c9-1f6d7741af62 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.590956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00a3cb-4c68-4a8b-b9cb-f7735928c4f2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.635105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.635742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.636016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.636287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] PATH_INFO: `/v3/users/2d916167c863408aa7c93c578c109e9d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:54.640323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] RBAC: Authorizing `identity:get_user(user_id=2d916167c863408aa7c93c578c109e9d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:54.641799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:54.649893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 142/289] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:54 2022] GET /identity/v3/users/2d916167c863408aa7c93c578c109e9d => generated 297 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:54.650754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:54.657667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.658329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.658606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.658872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:54.664616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:54.666099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:54.669021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 148/290] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:54 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:54.673697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:54.774096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:54.816577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d319f4-ca2d-4d8d-b58b-ca4f594f62d7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.843789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27ff91-cdac-4653-a917-f1dfa837a016 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:54.848341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:54.855090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.855747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.856023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.857444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/81810b1993724d9084304b2ae45da381/roles/969ed0ca7dc147ecad313e6a3909656d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:54.859944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] RBAC: Authorizing `identity:create_grant(project_id=883fcb9f0c0c404586e1b021177d762e, user_id=81810b1993724d9084304b2ae45da381, role_id=969ed0ca7dc147ecad313e6a3909656d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:54.861395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:54.881623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 149/291] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:58:54 2022] PUT /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e/users/81810b1993724d9084304b2ae45da381/roles/969ed0ca7dc147ecad313e6a3909656d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:54.919347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:54.919964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:54.920300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:54.920573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:55.020300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:55.021803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:55.102330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:55.152898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 143/292] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:55.159405 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:55.290713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4885-0c70-45ad-adac-a4a667e42d11 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:55.327301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63a3db-0cd0-432e-be75-edfbc1a79f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:55.327983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63a3db-0cd0-432e-be75-edfbc1a79f24 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:55.328215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63a3db-0cd0-432e-be75-edfbc1a79f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:55.328494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63a3db-0cd0-432e-be75-edfbc1a79f24 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:55.329616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63a3db-0cd0-432e-be75-edfbc1a79f24 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:55.331090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63a3db-0cd0-432e-be75-edfbc1a79f24 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:55.333107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 150/293] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:55 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:55.338532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:55.474155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65e740-3adf-45bf-94e3-a38177572ee8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:55.515985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300497c1-fc1f-4483-a64b-94f74f74175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:55.516594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300497c1-fc1f-4483-a64b-94f74f74175e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:55.516867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300497c1-fc1f-4483-a64b-94f74f74175e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:55.517132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300497c1-fc1f-4483-a64b-94f74f74175e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:55.517884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300497c1-fc1f-4483-a64b-94f74f74175e None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=2d916167c863408aa7c93c578c109e9d) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:55.518189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300497c1-fc1f-4483-a64b-94f74f74175e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:55.519702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300497c1-fc1f-4483-a64b-94f74f74175e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:55.534302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 144/294] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:55 2022] GET /identity/v3/role_assignments?user.id=2d916167c863408aa7c93c578c109e9d&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:55.739785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f406459-dcc0-497d-88f5-f4b7c16b9981 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:55.740129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f406459-dcc0-497d-88f5-f4b7c16b9981 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:55.740570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f406459-dcc0-497d-88f5-f4b7c16b9981 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:55.742516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 151/295] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:55.750146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f515a-e430-4b1d-bfa3-98abfbe1bd91 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:55.750441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f515a-e430-4b1d-bfa3-98abfbe1bd91 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:55.750693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f515a-e430-4b1d-bfa3-98abfbe1bd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:55.778844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1b6f515a-e430-4b1d-bfa3-98abfbe1bd91 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:55.780074 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b6f515a-e430-4b1d-bfa3-98abfbe1bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:55.809010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:55.860800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 145/296] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:56.099810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a256f-1d86-482d-a8ed-7e662bfea8ee None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.100825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a256f-1d86-482d-a8ed-7e662bfea8ee None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.101328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a256f-1d86-482d-a8ed-7e662bfea8ee None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.103581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 152/297] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:56.110298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dcb6a-a815-41af-9d66-b976a586b2ee None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.110818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dcb6a-a815-41af-9d66-b976a586b2ee None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.111027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dcb6a-a815-41af-9d66-b976a586b2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.137018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2e9dcb6a-a815-41af-9d66-b976a586b2ee None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:56.138758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e9dcb6a-a815-41af-9d66-b976a586b2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:56.157076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a256f-1d86-482d-a8ed-7e662bfea8ee None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.233033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:56.247461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 146/298] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:56.249542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e118dc-3427-4e47-8313-04fb7b2b4e38 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.250124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e118dc-3427-4e47-8313-04fb7b2b4e38 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.250526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e118dc-3427-4e47-8313-04fb7b2b4e38 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.252045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 147/299] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:56.259562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6486f1-e77b-4b52-b0b5-799e6698871b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.259845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6486f1-e77b-4b52-b0b5-799e6698871b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.260120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6486f1-e77b-4b52-b0b5-799e6698871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.284674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4a6486f1-e77b-4b52-b0b5-799e6698871b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:56.285912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a6486f1-e77b-4b52-b0b5-799e6698871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:56.287509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a256f-1d86-482d-a8ed-7e662bfea8ee None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:56.323329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:56.324240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.324704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.325147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.336556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:56.342278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:56.343277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:56.344065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:56.350982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 153/300] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:56.352005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.366078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 148/301] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:56.456501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604cb95-4902-46d7-94e7-1fb3012d6196 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:56.482060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:56.491393 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:56.491993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.492322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.492642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.493313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:56.493612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:56.495004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:56.504258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 154/302] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:56.509793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6486f1-e77b-4b52-b0b5-799e6698871b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.548457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.553584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:56.640967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6486f1-e77b-4b52-b0b5-799e6698871b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:56.662188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd1dbf-7500-40d8-8190-7274b9bafad1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:56.685477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:56.686507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.686930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.687370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] PATH_INFO: `/v3/users/03f962f0f47a4868aae769753ea79105` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.688616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] RBAC: Authorizing `identity:get_user(user_id=03f962f0f47a4868aae769753ea79105)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:56.690260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:56.691970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 149/303] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:56 2022] GET /identity/v3/users/03f962f0f47a4868aae769753ea79105 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:56.692867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.701665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:56.702394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.702908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.703290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.713500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:56.719291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:56.720136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 155/304] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:58:56.721555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.768763 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:56.795049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:56.805914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f35cb-38ce-4780-b8d8-637410dae54a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:56.833481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999bb70-aa8c-4dc7-9097-1583890b5026 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:56.841481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:56.842140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.842416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.842689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.845084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:56.846568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:56.848310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 150/305] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:56.848923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.876259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:56.876911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:56.877197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:56.877475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] PATH_INFO: `/v3/system/users/03f962f0f47a4868aae769753ea79105/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:56.878927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03f962f0f47a4868aae769753ea79105, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:56.880343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:56.899599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 156/306] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Sep 13 13:58:56 2022] PUT /identity/v3/system/users/03f962f0f47a4868aae769753ea79105/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:58:56.899599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:56.980443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041fec5-2e09-4c0f-a048-0638ac31ee4a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:57.024265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96fd40-3d05-4d76-9c42-246d1111e8ec None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:57.026505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:57.028393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:57.028760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:57.029111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:57.029924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:57.030344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:57.032049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:57.032754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:57.043282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 151/307] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:57.049993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:57.070765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:57.071794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:57.072211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:57.072623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] PATH_INFO: `/v3/groups` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:57.073461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:57.075396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:57.078516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:57.097267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 157/308] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Sep 13 13:58:56 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:58:57.185381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ea62b-18f5-4da1-a82d-6b7aded50d32 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:57.226139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c125c3-6410-49d5-ab4a-0c2e77a89d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:57.226797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c125c3-6410-49d5-ab4a-0c2e77a89d7f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:57.227081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c125c3-6410-49d5-ab4a-0c2e77a89d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:57.227355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c125c3-6410-49d5-ab4a-0c2e77a89d7f None admin] PATH_INFO: `/v3/users/0da1f5e5e58a433ab330107d416c4d72` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:57.243955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c125c3-6410-49d5-ab4a-0c2e77a89d7f None admin] RBAC: Authorizing `identity:get_user(user_id=0da1f5e5e58a433ab330107d416c4d72)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:57.245509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c125c3-6410-49d5-ab4a-0c2e77a89d7f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:57.247706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 152/309] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:57 2022] GET /identity/v3/users/0da1f5e5e58a433ab330107d416c4d72 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:57.254515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:57.377624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c2bf4-274a-47cf-9300-9134caddaa47 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:57.413849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758529b-318d-426b-8f88-25c04458d221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:57.414696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758529b-318d-426b-8f88-25c04458d221 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:57.415150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758529b-318d-426b-8f88-25c04458d221 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:57.415561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758529b-318d-426b-8f88-25c04458d221 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:57.416427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758529b-318d-426b-8f88-25c04458d221 None admin] RBAC: Adding query filter params (role.id=969ed0ca7dc147ecad313e6a3909656d, user.id=0da1f5e5e58a433ab330107d416c4d72, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:57.416872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758529b-318d-426b-8f88-25c04458d221 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:57.418465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758529b-318d-426b-8f88-25c04458d221 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:57.424487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 158/310] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 13 13:58:57 2022] GET /identity/v3/role_assignments?user.id=0da1f5e5e58a433ab330107d416c4d72&scope.system=all&role.id=969ed0ca7dc147ecad313e6a3909656d => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:57.463829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed62cf-c052-49a3-a09d-4654ddc9a7b0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:57.464188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed62cf-c052-49a3-a09d-4654ddc9a7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:57.464541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed62cf-c052-49a3-a09d-4654ddc9a7b0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:57.466123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 153/311] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:57.472354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7db7-df52-41f5-bb23-af74dc4a7afe None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:57.472653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7db7-df52-41f5-bb23-af74dc4a7afe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:57.472940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b7db7-df52-41f5-bb23-af74dc4a7afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:57.493642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-083b7db7-df52-41f5-bb23-af74dc4a7afe None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:57.494997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-083b7db7-df52-41f5-bb23-af74dc4a7afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:57.526343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:57.593884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 159/312] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:57.867462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed62cf-c052-49a3-a09d-4654ddc9a7b0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:57.954853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:57.979842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed62cf-c052-49a3-a09d-4654ddc9a7b0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:58.016416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:58.017097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:58.017377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:58.017644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:58.028472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:58.033854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:58.034725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:58:58.035523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:58.039766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 154/313] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:58.041272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7db7-df52-41f5-bb23-af74dc4a7afe None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:58.150734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b7db7-df52-41f5-bb23-af74dc4a7afe None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:58.187984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:58.188595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:58.188934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:58.189213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:58.189907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:58.190210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:58.191580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:58.204020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 160/314] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:58.206696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:58.318198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f7eb5-4961-4488-b278-735deaa0c43b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:58.355907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:58.356538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:58.356817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:58.357087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] PATH_INFO: `/v3/users/81810b1993724d9084304b2ae45da381` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:58.358240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=81810b1993724d9084304b2ae45da381)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:58.359738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:58.361635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 155/315] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:58 2022] GET /identity/v3/users/81810b1993724d9084304b2ae45da381 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:58.366712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:58.393122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:58.477643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee2930-3f2b-4517-b2ca-e4fe49363efe None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:58.512604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:58.513199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:58.513476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:58.513764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:58.594368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:58.595823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:58.619856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:58.701427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 161/316] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:58:58.707155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:58.715878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:58.817139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f9c71-6dfb-4464-bf7d-88e1d2408ca1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:58.850943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099cb64-d240-4948-b212-50133bacf748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:58.851574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cb64-d240-4948-b212-50133bacf748 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:58.851849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cb64-d240-4948-b212-50133bacf748 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:58.852120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099cb64-d240-4948-b212-50133bacf748 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:58.853252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099cb64-d240-4948-b212-50133bacf748 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:58.854711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099cb64-d240-4948-b212-50133bacf748 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:58.856649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 156/317] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:58:58 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:58.862777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:58.963818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5feaf3e-0022-4704-9613-a3fdd9ef3c12 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:58.987571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:59.000634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24e3f-7d12-4cbb-a919-964ec5fec006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:59.001445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24e3f-7d12-4cbb-a919-964ec5fec006 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.001758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24e3f-7d12-4cbb-a919-964ec5fec006 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.002020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24e3f-7d12-4cbb-a919-964ec5fec006 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.002741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24e3f-7d12-4cbb-a919-964ec5fec006 None admin] RBAC: Adding query filter params (role.id=969ed0ca7dc147ecad313e6a3909656d, scope.project.id=883fcb9f0c0c404586e1b021177d762e, user.id=81810b1993724d9084304b2ae45da381) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:59.003044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24e3f-7d12-4cbb-a919-964ec5fec006 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:59.004458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24e3f-7d12-4cbb-a919-964ec5fec006 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:59.010531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 162/318] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:58:58 2022] GET /identity/v3/role_assignments?user.id=81810b1993724d9084304b2ae45da381&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=969ed0ca7dc147ecad313e6a3909656d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:58:59.231388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427328-bd2d-4437-902a-827b70e29f87 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.232251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427328-bd2d-4437-902a-827b70e29f87 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.232751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6427328-bd2d-4437-902a-827b70e29f87 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.234667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 157/319] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:59.240136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126af19-20d1-45b2-9d15-8be412d7cae3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.240669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126af19-20d1-45b2-9d15-8be412d7cae3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.241139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126af19-20d1-45b2-9d15-8be412d7cae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.261108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5126af19-20d1-45b2-9d15-8be412d7cae3 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:59.262723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5126af19-20d1-45b2-9d15-8be412d7cae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:59.331686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 163/320] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:59.449956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b6fbb-f57a-40a4-9d9c-809c7d8a722c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.450232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b6fbb-f57a-40a4-9d9c-809c7d8a722c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.450692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b6fbb-f57a-40a4-9d9c-809c7d8a722c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.452457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 158/321] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:58:59.458009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce02e6-31e2-428d-8b68-2ee346d76158 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.458446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce02e6-31e2-428d-8b68-2ee346d76158 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.458555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce02e6-31e2-428d-8b68-2ee346d76158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.476501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a8ce02e6-31e2-428d-8b68-2ee346d76158 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:59.477677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a8ce02e6-31e2-428d-8b68-2ee346d76158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:59.551676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 164/322] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:58:59.604903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b6fbb-f57a-40a4-9d9c-809c7d8a722c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:59.667712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8b11-abbb-4950-8b65-515041523efb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.668003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8b11-abbb-4950-8b65-515041523efb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.668282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce8b11-abbb-4950-8b65-515041523efb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.669621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 165/323] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:58:59.675406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f53da4-4ea0-48e4-b1e8-8e1fee0683e3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.678527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f53da4-4ea0-48e4-b1e8-8e1fee0683e3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.678527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f53da4-4ea0-48e4-b1e8-8e1fee0683e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.685215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:59.694640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-57f53da4-4ea0-48e4-b1e8-8e1fee0683e3 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:58:59.695831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57f53da4-4ea0-48e4-b1e8-8e1fee0683e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:58:59.714111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b6fbb-f57a-40a4-9d9c-809c7d8a722c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:59.740610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:59.750588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:59.751218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.751502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.751777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.762433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:58:59.767797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:58:59.768673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:58:59.769490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 159/324] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:58:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:58:59.771766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 166/325] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:58:59.774638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:59.816951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f53da4-4ea0-48e4-b1e8-8e1fee0683e3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:59.879969 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c83b0-8c57-4f31-b8b8-886952c57093 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:59.915622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:59.916227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.916509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.916787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.917452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:58:59.917768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:59.919165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:59.921894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:58:59.923658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f53da4-4ea0-48e4-b1e8-8e1fee0683e3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:58:59.929413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 160/326] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:58:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:59.934945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:58:59.963244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:58:59.963775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:58:59.964068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:58:59.964318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:58:59.965412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:58:59.966872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:58:59.969759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 167/327] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:58:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:58:59.974352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:00.021222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:00.041978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3735f-1670-4788-be18-069a6d1c1ecc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:00.079186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9b899-68ff-4ae1-a995-b8a2a8d749e5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:00.080478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:00.081107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:00.081386 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:00.081657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] PATH_INFO: `/v3/users/03f962f0f47a4868aae769753ea79105` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:00.082813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] RBAC: Authorizing `identity:get_user(user_id=03f962f0f47a4868aae769753ea79105)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:00.084246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:00.085843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 161/328] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:58:59 2022] GET /identity/v3/users/03f962f0f47a4868aae769753ea79105 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:00.087443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:00.112718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:00.113301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:00.113581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:00.113885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] PATH_INFO: `/v3/groups` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:00.114509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:00.115921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:00.130969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 168/329] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Sep 13 13:58:59 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 13:59:00.134718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:00.156455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:00.200401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bd4fe-f967-4d9b-8966-572af0ff8333 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:00.239640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:00.241125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:00.241857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:00.242300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:00.255951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:00.257868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92069fa5-2ad7-41a8-a376-d981c8bcf3ee None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:00.261550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:00.262728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:59:00.266838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:00.267600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:00.267711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 162/330] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:00.269965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:00.305131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:00.305471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:00.305656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:00.305964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:00.306704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] RBAC: Adding query filter params (role.id=6bc93361bf6e4aeb91fd6c5713557177, user.id=03f962f0f47a4868aae769753ea79105, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:00.307012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:00.308436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:00.321311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 169/331] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 13 13:59:00 2022] GET /identity/v3/role_assignments?user.id=03f962f0f47a4868aae769753ea79105&scope.system=all&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:00.393937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502846-c918-4894-a847-634a61e6ea7f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:00.424386 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:00.437504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268387-abda-4c72-9384-22771989fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:00.438190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268387-abda-4c72-9384-22771989fc95 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:00.438465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268387-abda-4c72-9384-22771989fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:00.438814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268387-abda-4c72-9384-22771989fc95 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:00.439474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68268387-abda-4c72-9384-22771989fc95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:00.439771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68268387-abda-4c72-9384-22771989fc95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:00.441197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68268387-abda-4c72-9384-22771989fc95 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:00.451735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 163/332] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:00.458493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:00.580094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bb5be-0f87-4a7c-aa41-838fd940ea70 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:00.617566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9c38e-fe18-4581-8da8-a2e0deaa4ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:00.618250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9c38e-fe18-4581-8da8-a2e0deaa4ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:00.618519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9c38e-fe18-4581-8da8-a2e0deaa4ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:00.618794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9c38e-fe18-4581-8da8-a2e0deaa4ef7 None admin] PATH_INFO: `/v3/users/0da1f5e5e58a433ab330107d416c4d72` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:00.619955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9c38e-fe18-4581-8da8-a2e0deaa4ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=0da1f5e5e58a433ab330107d416c4d72)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:00.621381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9c38e-fe18-4581-8da8-a2e0deaa4ef7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:00.623290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 170/333] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:59:00 2022] GET /identity/v3/users/0da1f5e5e58a433ab330107d416c4d72 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:00.629550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268387-abda-4c72-9384-22771989fc95 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:00.742803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268387-abda-4c72-9384-22771989fc95 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:00.783632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ccec4-b52a-423d-86ff-33c883247304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:00.784298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ccec4-b52a-423d-86ff-33c883247304 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:00.784580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ccec4-b52a-423d-86ff-33c883247304 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:00.784857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ccec4-b52a-423d-86ff-33c883247304 None admin] PATH_INFO: `/v3/system/users/0da1f5e5e58a433ab330107d416c4d72/roles/969ed0ca7dc147ecad313e6a3909656d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:00.786342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ccec4-b52a-423d-86ff-33c883247304 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0da1f5e5e58a433ab330107d416c4d72, role_id=969ed0ca7dc147ecad313e6a3909656d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:00.787741 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ccec4-b52a-423d-86ff-33c883247304 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:00.799302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 164/334] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Sep 13 13:59:00 2022] PUT /identity/v3/system/users/0da1f5e5e58a433ab330107d416c4d72/roles/969ed0ca7dc147ecad313e6a3909656d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:59:02.612078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a92a-92e9-4a86-b7fe-313688acb52a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:02.612574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a92a-92e9-4a86-b7fe-313688acb52a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:02.612574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688a92a-92e9-4a86-b7fe-313688acb52a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:02.613972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 171/335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:02.619391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d5a6-833d-4f93-8355-20ed0f535841 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:02.619850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d5a6-833d-4f93-8355-20ed0f535841 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:02.620026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47d5a6-833d-4f93-8355-20ed0f535841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:02.646824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-af47d5a6-833d-4f93-8355-20ed0f535841 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:02.648033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-af47d5a6-833d-4f93-8355-20ed0f535841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:02.710524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:02.744237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 165/336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:02.902899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542add0-196e-4f26-82e4-cc004546f058 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:02.903192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542add0-196e-4f26-82e4-cc004546f058 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:02.903356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542add0-196e-4f26-82e4-cc004546f058 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:02.904906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 172/337] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:02.912238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7a4d-1cbc-4fc8-b68b-35df87292e9e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:02.912573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7a4d-1cbc-4fc8-b68b-35df87292e9e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:02.912854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e7a4d-1cbc-4fc8-b68b-35df87292e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:02.942913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3a2e7a4d-1cbc-4fc8-b68b-35df87292e9e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:02.944179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3a2e7a4d-1cbc-4fc8-b68b-35df87292e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:03.032888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 166/338] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:03.057430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6840-3ff8-4534-ba7e-fbe1b8ba220c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:03.057720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6840-3ff8-4534-ba7e-fbe1b8ba220c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:03.058004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a6840-3ff8-4534-ba7e-fbe1b8ba220c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:03.059594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 173/339] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:03.075513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d2182-a233-45a9-94ce-2aa7ec3b012a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:03.075800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d2182-a233-45a9-94ce-2aa7ec3b012a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:03.076074 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d2182-a233-45a9-94ce-2aa7ec3b012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:03.100095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bd7d2182-a233-45a9-94ce-2aa7ec3b012a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:03.101290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bd7d2182-a233-45a9-94ce-2aa7ec3b012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:03.188438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 167/340] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:03.216480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a6840-3ff8-4534-ba7e-fbe1b8ba220c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:03.263025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:03.349536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a6840-3ff8-4534-ba7e-fbe1b8ba220c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:03.387811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:03.388775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:03.389292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:03.389821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:03.406744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:03.412477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:03.413336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:03.414152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:03.416415 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 174/341] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:03.422293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d2182-a233-45a9-94ce-2aa7ec3b012a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:03.445516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:03.495451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:03.599310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:03.612058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12fd96-feca-4246-aa1f-c86c37c6185c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:03.623096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d2182-a233-45a9-94ce-2aa7ec3b012a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:03.669874 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:03.671494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:03.672109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:03.673364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:03.700007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:03.701404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:03.702254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:03.702989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:03.703480 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:03.708740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:03.709273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:03.714951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:03.719645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:03.720535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:03.721320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:03.724325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 175/342] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:59:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:03.730521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:03.738526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 168/343] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:03.739724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:03.868375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a704700-c847-4c1f-b0c0-0eb3294b4f92 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:03.909429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:03.910680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:03.911158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:03.911623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:03.912509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:03.913015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:03.915200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:03.929552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 176/344] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:03.931948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:03.973852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05fb80-0370-486c-a480-67a5af81aa35 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:03.976993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:03.998218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:04.060491 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:04.062699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.063448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.064218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.084313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:04.099317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:04.100283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:04.101162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:04.103712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 169/345] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:04.107680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:04.226934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6bc5b-684c-427c-8dfa-b3f555b0d6b8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:04.284409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:04.285444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.285943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.286408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.293276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd34ec-1a77-4570-b3bc-f9da7557cffc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:04.368569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:04.370962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:04.371937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0e877-87d0-4d0e-a790-bac341900ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:04.372780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0e877-87d0-4d0e-a790-bac341900ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.373254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0e877-87d0-4d0e-a790-bac341900ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.373719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0e877-87d0-4d0e-a790-bac341900ba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.387717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:04.414238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:04.455360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0e877-87d0-4d0e-a790-bac341900ba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:04.457276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0e877-87d0-4d0e-a790-bac341900ba0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:04.489202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 177/346] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:04.490154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:04.581678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 170/347] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:04.583030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f3ee4-1171-41dd-af54-dc29c41179f7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.583311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f3ee4-1171-41dd-af54-dc29c41179f7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.583581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f3ee4-1171-41dd-af54-dc29c41179f7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.584950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 171/348] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:04.585490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f3ee4-1171-41dd-af54-dc29c41179f7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:04.609762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d49b47-65f6-4209-a3ce-7832a5f26b55 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:04.634400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:04.651280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:04.652771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.653150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.653766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.654515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:04.654892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:04.656617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:04.661490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:04.666631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 178/349] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:04.668136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:04.713821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f3ee4-1171-41dd-af54-dc29c41179f7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:04.757991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f3625-c7d2-443f-b3dc-82e3b44372c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:04.758705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f3625-c7d2-443f-b3dc-82e3b44372c6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.758983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f3625-c7d2-443f-b3dc-82e3b44372c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.760037 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f3625-c7d2-443f-b3dc-82e3b44372c6 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.761910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f3625-c7d2-443f-b3dc-82e3b44372c6 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:04.765409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f3625-c7d2-443f-b3dc-82e3b44372c6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:04.767601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 172/350] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:04 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:04.769502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49e03-fd33-4454-bfe6-f78f2960ae23 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.769824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49e03-fd33-4454-bfe6-f78f2960ae23 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.770101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a49e03-fd33-4454-bfe6-f78f2960ae23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.793685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-70a49e03-fd33-4454-bfe6-f78f2960ae23 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:04.795092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70a49e03-fd33-4454-bfe6-f78f2960ae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:04.799061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658f68-c24f-4b4b-a010-b2b55aa813f5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:04.839569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:04.841037 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:04.841317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:04.841587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:04.842784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:04.844254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:04.853131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 179/351] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:04 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:04.853131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:04.878225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 173/352] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:04.879289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a49e03-fd33-4454-bfe6-f78f2960ae23 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:04.906478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:04.956713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:04.977341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa6272-f649-4bfd-af3c-16d9ec80d15d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.011162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a49e03-fd33-4454-bfe6-f78f2960ae23 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.018114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.018960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.019419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.019878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.057123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97995c68-c763-475d-818b-78c12a8ef3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.057781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97995c68-c763-475d-818b-78c12a8ef3bb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.058118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97995c68-c763-475d-818b-78c12a8ef3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.058735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97995c68-c763-475d-818b-78c12a8ef3bb None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.068025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97995c68-c763-475d-818b-78c12a8ef3bb None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:05.069472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97995c68-c763-475d-818b-78c12a8ef3bb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:05.071690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 174/353] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:04 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:05.073467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e224c83-3545-4d17-9a0d-a0ff5c62c093 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.073763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e224c83-3545-4d17-9a0d-a0ff5c62c093 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.074038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e224c83-3545-4d17-9a0d-a0ff5c62c093 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.075655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 175/354] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:05.076726 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e224c83-3545-4d17-9a0d-a0ff5c62c093 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:05.106188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:05.108170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:05.179209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:05.232551 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e224c83-3545-4d17-9a0d-a0ff5c62c093 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.254690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 180/355] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:05.255503 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:05.269398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:05.279027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0396ba-34b4-40a0-a79c-fb9cb25720f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.279662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0396ba-34b4-40a0-a79c-fb9cb25720f7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.279945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0396ba-34b4-40a0-a79c-fb9cb25720f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.280262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0396ba-34b4-40a0-a79c-fb9cb25720f7 None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.281496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0396ba-34b4-40a0-a79c-fb9cb25720f7 None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:05.282988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0396ba-34b4-40a0-a79c-fb9cb25720f7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:05.284557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 176/356] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:05.285713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29760a-2e33-4c1e-a986-9f6d1c18bc0f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.285985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29760a-2e33-4c1e-a986-9f6d1c18bc0f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.286258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29760a-2e33-4c1e-a986-9f6d1c18bc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.306373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cb29760a-2e33-4c1e-a986-9f6d1c18bc0f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:05.307166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cb29760a-2e33-4c1e-a986-9f6d1c18bc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:05.380021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7fb5-d8ed-4299-9f4d-c137677d795f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.386294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 177/357] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:05.387236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29760a-2e33-4c1e-a986-9f6d1c18bc0f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:05.421745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.422360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.422882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.422882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.423636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=18b7b138b42148d79097c7b2e243b955) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:05.423935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:05.425514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:05.438120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 181/358] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:05.445920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:05.470738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:05.510193 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29760a-2e33-4c1e-a986-9f6d1c18bc0f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.547580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:05.566169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903269c-012f-43f4-89a2-0dc48060330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.567106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903269c-012f-43f4-89a2-0dc48060330d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.567390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903269c-012f-43f4-89a2-0dc48060330d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.567663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903269c-012f-43f4-89a2-0dc48060330d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.586847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903269c-012f-43f4-89a2-0dc48060330d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:05.592164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2732e-d78d-40b1-96ad-084d1ff159da None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6903269c-012f-43f4-89a2-0dc48060330d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:05.593439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:05.594359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 13:59:05.595184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:05.599790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 178/359] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:05.602450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673247-f88d-44e0-8938-d0bbe0fa2846 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.602740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673247-f88d-44e0-8938-d0bbe0fa2846 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.603019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae673247-f88d-44e0-8938-d0bbe0fa2846 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.605137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 179/360] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:05.606412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae673247-f88d-44e0-8938-d0bbe0fa2846 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:05.659795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.660483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.661049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.661049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.662203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:05.663635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:05.665296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 182/361] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:05.669752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:05.753043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae673247-f88d-44e0-8938-d0bbe0fa2846 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.762432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:05.797982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2519-8e72-44b9-b683-e432a0dfe161 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:05.803283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fd253-5fc0-479d-9612-0e5930184bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.803898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fd253-5fc0-479d-9612-0e5930184bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.804176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fd253-5fc0-479d-9612-0e5930184bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.804448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fd253-5fc0-479d-9612-0e5930184bd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.805179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fd253-5fc0-479d-9612-0e5930184bd7 None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=18b7b138b42148d79097c7b2e243b955) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:05.805485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fd253-5fc0-479d-9612-0e5930184bd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:05.806982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fd253-5fc0-479d-9612-0e5930184bd7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:05.814632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 180/362] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:05.815871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591d31-329e-477f-a067-721190066f24 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.816157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591d31-329e-477f-a067-721190066f24 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.816432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591d31-329e-477f-a067-721190066f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.846877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-db591d31-329e-477f-a067-721190066f24 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:05.848219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db591d31-329e-477f-a067-721190066f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:05.848793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:05.849422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:05.849698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:05.850002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:05.850676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:05.850983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:05.852365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:05.868779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 183/363] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:05.869812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:05.909351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:05.947313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 181/364] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:05.948238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591d31-329e-477f-a067-721190066f24 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.005199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c38db0-05cc-4caf-a37f-873e529ce98c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.056601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.057143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.057418 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.057686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.058867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:06.060318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:06.067008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 184/365] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:06.067008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.074311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:06.127798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591d31-329e-477f-a067-721190066f24 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.182984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.183628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.183907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.184175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.207129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:06.212165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:06.213026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:06.213855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:06.219043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 182/366] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:59:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:06.220051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.228655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55629cc-8576-4a3c-bfa8-f8cadee87e45 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.258569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:06.313209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.314176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.314630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.315077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] PATH_INFO: `/v3/users/0da1f5e5e58a433ab330107d416c4d72` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.316395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] RBAC: Authorizing `identity:get_user(user_id=0da1f5e5e58a433ab330107d416c4d72)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:06.318019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:06.324484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 185/367] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 13:59:06 2022] GET /identity/v3/users/0da1f5e5e58a433ab330107d416c4d72 => generated 293 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:06.325391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.336779 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6adee-7e14-48d4-bc87-cb42550e4573 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.386369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.387849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.388595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.389070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.390304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=883fcb9f0c0c404586e1b021177d762e) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:06.391238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:06.393834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:06.401619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 183/368] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:06 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:06.403539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.408033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:06.451329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6ef9-90b5-4d53-8705-30870a9223d7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.496661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.497291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.497567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.497870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.498613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:06.498923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:06.500589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:06.509706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 186/369] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:06.510516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.583078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:06.592316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943ed1-2d23-436c-a73a-2e5e7d72c12b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.646951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829323-7e5b-4c03-8cf8-9f5f5cefd3f6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.660133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.660133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.660133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.660133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.678741 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:06.689172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:06.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:06.691126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:06.695012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.695814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.696236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.696650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.697521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] RBAC: Adding query filter params (role.id=969ed0ca7dc147ecad313e6a3909656d, user.id=0da1f5e5e58a433ab330107d416c4d72, scope.system=all) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:06.697943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 184/370] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:06.698962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.699424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:06.701012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:06.706606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:06.728517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 187/371] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Sep 13 13:59:06 2022] GET /identity/v3/role_assignments?user.id=0da1f5e5e58a433ab330107d416c4d72&scope.system=all&role.id=969ed0ca7dc147ecad313e6a3909656d => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:06.729666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:06.873770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3065ea3-0ee5-4ce6-ac72-5c153db83595 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:06.925708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:06.926751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:06.927405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:06.928808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:06.989565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f232e60-b0cf-40c6-820a-48b2c2452fac None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:07.002101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:07.036919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:07.038142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:07.038427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:07.038701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:07.039394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:07.039701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:07.041300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:07.051441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 188/372] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:07.058387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:07.072205 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:07.074189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:07.125049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:07.181338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644fc81-08c2-48e2-9782-4306fd917e2e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:07.220805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:07.221414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:07.221697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:07.222003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:07.261355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 185/373] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:07.267358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:07.302660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:07.305750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:07.367041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:07.397857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:07.419068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 189/374] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 13:59:07.425453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:07.454144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb54bc-0511-4356-8e7a-ddad847c201a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:07.516324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:07.517409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:07.517988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:07.518490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:07.519834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:07.521639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:07.523862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 186/375] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:07 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:07.536891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d5518-803f-449e-8b10-25a88c7b0043 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:07.544816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:07.589007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:07.589837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:07.590124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:07.590398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] PATH_INFO: `/v3/projects/4d23d643906b4c84bfbe35da532a7c31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:07.591586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] RBAC: Authorizing `identity:get_project(project_id=4d23d643906b4c84bfbe35da532a7c31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:07.593366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:07.595885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 190/376] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:07 2022] GET /identity/v3/projects/4d23d643906b4c84bfbe35da532a7c31 => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:07.602543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:07.632439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:07.760192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf9e3d-bab3-4c28-93db-649eb017ac3f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:07.794018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabdac8-8234-4eae-9bd8-2f1e9a7ca303 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:07.816473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:07.841518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:07.842192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:07.842473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:07.842771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] PATH_INFO: `/v3/groups/8829a4e6f66346ba93b72b9801c5b9e7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:07.846493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:07.847153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:07.847429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:07.847704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:07.848865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:07.850351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:07.851536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] RBAC: Authorizing `identity:get_group(group_id=8829a4e6f66346ba93b72b9801c5b9e7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:07.852215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 187/377] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:07 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:07.853336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:07.855409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 191/378] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:07 2022] GET /identity/v3/groups/8829a4e6f66346ba93b72b9801c5b9e7 => generated 229 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:07.857851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:07.861332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:07.966214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:07.972952 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f46213-1eda-40db-b9a0-d5730e3e3497 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:08.010706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249256b5-e2eb-4681-8570-a32f4614ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:08.012018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249256b5-e2eb-4681-8570-a32f4614ed23 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:08.012671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249256b5-e2eb-4681-8570-a32f4614ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:08.013585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249256b5-e2eb-4681-8570-a32f4614ed23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:08.015164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249256b5-e2eb-4681-8570-a32f4614ed23 None admin] RBAC: Adding query filter params (group.id=8829a4e6f66346ba93b72b9801c5b9e7, role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=4d23d643906b4c84bfbe35da532a7c31) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:08.015819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249256b5-e2eb-4681-8570-a32f4614ed23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:08.018300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249256b5-e2eb-4681-8570-a32f4614ed23 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:08.031496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 192/379] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:07 2022] GET /identity/v3/role_assignments?group.id=8829a4e6f66346ba93b72b9801c5b9e7&scope.project.id=4d23d643906b4c84bfbe35da532a7c31&role.id=96af379f1477489b8542698d3c012f1a => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:08.087950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d849f-9cb4-4aec-a889-b28272631c2e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:08.141807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671e95d-2387-4472-b4c8-0b52c230ef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:08.142432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e95d-2387-4472-b4c8-0b52c230ef7b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:08.142713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e95d-2387-4472-b4c8-0b52c230ef7b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:08.142979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671e95d-2387-4472-b4c8-0b52c230ef7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:08.143786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e95d-2387-4472-b4c8-0b52c230ef7b None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=883fcb9f0c0c404586e1b021177d762e) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:08.144094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e95d-2387-4472-b4c8-0b52c230ef7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:08.145537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671e95d-2387-4472-b4c8-0b52c230ef7b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:08.152163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 188/380] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:07 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 260 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:08.532326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9072f-3729-4d31-8162-c81d1ef6a326 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:08.532881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9072f-3729-4d31-8162-c81d1ef6a326 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:08.533374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9072f-3729-4d31-8162-c81d1ef6a326 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:08.535199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 193/381] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:08.540971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0b658-45d7-46c0-b5aa-57f7356911ce None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:08.541522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0b658-45d7-46c0-b5aa-57f7356911ce None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:08.542038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0b658-45d7-46c0-b5aa-57f7356911ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:08.575073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d0a0b658-45d7-46c0-b5aa-57f7356911ce None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:08.576625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0a0b658-45d7-46c0-b5aa-57f7356911ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:08.644670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:08.663390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 189/382] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:08.859437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5559-d5a6-4c8d-b750-12cfd086dff7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:08.860089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5559-d5a6-4c8d-b750-12cfd086dff7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:08.860576 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5559-d5a6-4c8d-b750-12cfd086dff7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:08.862141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 194/383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:59:08.867694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f842796-ceb8-4faa-a58b-677564dc624a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:08.868247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f842796-ceb8-4faa-a58b-677564dc624a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:08.868752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f842796-ceb8-4faa-a58b-677564dc624a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:08.892179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2f842796-ceb8-4faa-a58b-677564dc624a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:08.893687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f842796-ceb8-4faa-a58b-677564dc624a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:08.989176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 190/384] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:09.066796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79276b-a4ab-4881-a522-e21745f6fd43 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:09.067413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79276b-a4ab-4881-a522-e21745f6fd43 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:09.067413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79276b-a4ab-4881-a522-e21745f6fd43 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:09.068661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 195/385] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:09.074149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b381f-2212-49ad-b099-485611fff718 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:09.074597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b381f-2212-49ad-b099-485611fff718 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:09.075040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b381f-2212-49ad-b099-485611fff718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:09.098241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ab3b381f-2212-49ad-b099-485611fff718 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:09.100081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab3b381f-2212-49ad-b099-485611fff718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:09.186300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 191/386] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:09.255790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79276b-a4ab-4881-a522-e21745f6fd43 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:09.386021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79276b-a4ab-4881-a522-e21745f6fd43 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:09.425275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:09.426309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:09.426810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:09.427304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:09.458263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:09.463630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:09.464559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:09.465396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:09.470983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 196/387] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:09.476935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b381f-2212-49ad-b099-485611fff718 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:09.491941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:09.507468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:09.566117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:09.628443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b381f-2212-49ad-b099-485611fff718 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:09.632376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f8d81-a6ea-409e-b6ee-05c6b5b2b773 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:09.669935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:09.672053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:09.672586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:09.672961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:09.673789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:09.674398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:09.675755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:09.676147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:09.676725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:09.677152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:09.677574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:09.688060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 192/388] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:09.689049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:09.691241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:09.696979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:09.698104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:09.699087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:09.703441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 197/389] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:59:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:09.704510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:09.797018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:09.827960 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546c711-e24c-4270-a802-def74c73b68a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:09.836681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8ece-7fb8-4df2-967d-ff0a7209dc0a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:09.842397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:09.865806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:09.866441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:09.866753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:09.867132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:09.878171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:09.879317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:09.879742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:09.880216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:09.880695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:09.882062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:09.883736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:09.885363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:09.886288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:09.887091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 193/390] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:09.887797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:09.888328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 198/391] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:09 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:09.892981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.000760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37c97d-9092-4718-bb12-ec3afd225231 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.008822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c6b4-c75d-4159-adcd-131dd668402d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.038008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.038704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.039055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.039404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.040235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:10.040710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.042242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.051701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 194/392] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:10.052782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.061059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.062427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.062938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.063422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.064290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:10.064846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.068817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.075484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:10.082862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 199/393] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:10.084041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.095879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:10.179410 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc76aa-1c6c-4fed-8d41-7ad75b978cc8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.235958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b15b1-5961-4483-9f84-2034df17fd13 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.236695 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.237377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.237660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.237981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.239326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.241674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.243665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 195/394] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:10 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:10.244449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.286865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.288218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.288560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.288841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.289985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.292551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.294473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 200/395] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:10 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:10.295231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.386139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa51c-f822-4b42-8516-53c21ecdd12e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.387842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:10.394216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:10.434489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.435462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.436012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.436329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.438687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.440145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.442021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8731a9c-4234-40f1-abf1-314519d0186e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.442137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 196/396] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:10 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:10.442834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.498122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.499752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.500323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.500608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.502592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] RBAC: Authorizing `identity:create_grant(project_id=18b7b138b42148d79097c7b2e243b955, group_id=762554bb5ca74134b58d9b5a3dd2a5b5, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.504591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.531891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 201/397] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 13 13:59:10 2022] PUT /identity/v3/projects/18b7b138b42148d79097c7b2e243b955/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:10.532932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.591791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd804f-ebaf-497b-8870-337c2f334d77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.666938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.669015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.669452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.669952 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.672772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.675085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.680823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 197/398] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:10 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:10.687406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.736982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead55ea6-a935-4458-a477-68c60091e58c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.771555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:10.776522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:10.789068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.790050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.790501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.791280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.792728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.794448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.798536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 202/399] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:10 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:10.805188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:10.871025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73061da-0a69-4049-bc92-df45819ea154 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.912521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63b28b-84bb-4055-a12c-4f89bba01d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.913538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63b28b-84bb-4055-a12c-4f89bba01d53 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.913844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63b28b-84bb-4055-a12c-4f89bba01d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.914119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63b28b-84bb-4055-a12c-4f89bba01d53 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.916845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63b28b-84bb-4055-a12c-4f89bba01d53 None admin] RBAC: Authorizing `identity:create_grant(project_id=18b7b138b42148d79097c7b2e243b955, group_id=762554bb5ca74134b58d9b5a3dd2a5b5, role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.919003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63b28b-84bb-4055-a12c-4f89bba01d53 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:10.941774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 198/400] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 13 13:59:10 2022] PUT /identity/v3/projects/18b7b138b42148d79097c7b2e243b955/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:59:10.941774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2358a-8a0f-4c19-a27d-2c301d337c02 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.941774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2358a-8a0f-4c19-a27d-2c301d337c02 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.942144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36ab37-104e-4d97-8164-2643084fc6c2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:10.942232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2358a-8a0f-4c19-a27d-2c301d337c02 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.942232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 199/401] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:10.942232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79323-a6e9-40a8-9519-78f6d9df6b62 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.942232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79323-a6e9-40a8-9519-78f6d9df6b62 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.942232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f79323-a6e9-40a8-9519-78f6d9df6b62 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.943020 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 200/402] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:10.948689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345b956-37fe-48b1-932d-4ab5ab5282d4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.948975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345b956-37fe-48b1-932d-4ab5ab5282d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.949245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345b956-37fe-48b1-932d-4ab5ab5282d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.973945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a345b956-37fe-48b1-932d-4ab5ab5282d4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:10.975385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a345b956-37fe-48b1-932d-4ab5ab5282d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:10.985017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5ec06-ffae-49ec-afa3-66927fcdef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:10.986097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5ec06-ffae-49ec-afa3-66927fcdef57 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:10.986579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5ec06-ffae-49ec-afa3-66927fcdef57 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:10.987355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5ec06-ffae-49ec-afa3-66927fcdef57 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:10.990316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5ec06-ffae-49ec-afa3-66927fcdef57 None admin] RBAC: Authorizing `identity:create_grant(project_id=883fcb9f0c0c404586e1b021177d762e, group_id=762554bb5ca74134b58d9b5a3dd2a5b5, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:10.992090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5ec06-ffae-49ec-afa3-66927fcdef57 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:11.009930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 203/403] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 13 13:59:10 2022] PUT /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:11.018266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec55bb-b440-4b67-a0c3-7934c3e03b9f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:11.018585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec55bb-b440-4b67-a0c3-7934c3e03b9f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:11.018848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec55bb-b440-4b67-a0c3-7934c3e03b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:11.058993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-beec55bb-b440-4b67-a0c3-7934c3e03b9f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:11.060243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-beec55bb-b440-4b67-a0c3-7934c3e03b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:11.088656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 201/404] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:11.126160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:11.148470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 204/405] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:11.406486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345b956-37fe-48b1-932d-4ab5ab5282d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:11.445425 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:11.534411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec55bb-b440-4b67-a0c3-7934c3e03b9f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:11.562090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345b956-37fe-48b1-932d-4ab5ab5282d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:11.604905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:11.605605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:11.605928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:11.606196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:11.627185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:11.632054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:11.632979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:11.633817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 202/406] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:59:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:11.642357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:11.697384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec55bb-b440-4b67-a0c3-7934c3e03b9f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:11.751608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:11.752257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:11.752534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:11.752806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:11.776661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:11.789318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:11.790291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:11.791111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 205/407] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:11.797700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:11.822777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:11.837188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ee174-0374-40dd-8aea-4c3bdc3dda41 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:11.878397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:11.879005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:11.879281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:11.879548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:11.880206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:11.880501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:11.881903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:11.900584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 203/408] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:11.913780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:11.915389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:12.024610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d288d-28ad-437a-9a36-f95132af5c34 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:12.034612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d007897-fb82-4bbc-8b42-9d5808341230 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:12.063226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:12.064306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:12.064854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:12.065324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:12.066822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:12.068658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:12.075092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 204/409] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:11 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:12.076577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:12.091313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:12.091835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:12.092117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:12.092457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:12.093243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:12.093578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:12.095491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:12.100291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:12.106677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 206/410] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:12.119586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:12.269990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2f172-4bc5-42e8-a932-1b27d961fb7b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:12.340719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:12.342310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:12.343476 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:12.344667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:12.345913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f1c71-bd1a-4ab8-ab19-ee9015101833 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:12.348782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:12.350377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:12.364316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 205/411] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:12 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:12.368724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:12.382853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:12.431163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:12.432440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:12.433763 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:12.434297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] PATH_INFO: `/v3/groups/8829a4e6f66346ba93b72b9801c5b9e7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:12.437654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] RBAC: Authorizing `identity:get_group(group_id=8829a4e6f66346ba93b72b9801c5b9e7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:12.440012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:12.442636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 207/412] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:12 2022] GET /identity/v3/groups/8829a4e6f66346ba93b72b9801c5b9e7 => generated 229 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:12.448062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:12.537068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:12.569085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5da3bd-ddff-4525-9aed-ff51c732bb50 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:12.645827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:12.646416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:12.647055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:12.647336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:12.649652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] RBAC: Authorizing `identity:create_grant(project_id=883fcb9f0c0c404586e1b021177d762e, group_id=762554bb5ca74134b58d9b5a3dd2a5b5, role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:12.651440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:12.662568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dd317-e671-4926-987f-7c96a631a6d1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:12.679695 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 206/413] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 13 13:59:12 2022] PUT /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e/groups/762554bb5ca74134b58d9b5a3dd2a5b5/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:12.719943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b0d79-6ac4-4088-abd2-05f806accc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:12.720655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b0d79-6ac4-4088-abd2-05f806accc3e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:12.720947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b0d79-6ac4-4088-abd2-05f806accc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:12.721231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b0d79-6ac4-4088-abd2-05f806accc3e None admin] PATH_INFO: `/v3/projects/4d23d643906b4c84bfbe35da532a7c31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:12.722466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b0d79-6ac4-4088-abd2-05f806accc3e None admin] RBAC: Authorizing `identity:get_project(project_id=4d23d643906b4c84bfbe35da532a7c31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:12.724550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b0d79-6ac4-4088-abd2-05f806accc3e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:12.726664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 208/414] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:12 2022] GET /identity/v3/projects/4d23d643906b4c84bfbe35da532a7c31 => generated 344 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:12.732638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:12.892980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad276c-cf67-47c6-a27b-1a6962daa797 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:12.930204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8e317-a186-45e5-96a3-5534410709a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:12.930773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8e317-a186-45e5-96a3-5534410709a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:12.931049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8e317-a186-45e5-96a3-5534410709a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:12.931325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8e317-a186-45e5-96a3-5534410709a6 None admin] PATH_INFO: `/v3/projects/4d23d643906b4c84bfbe35da532a7c31/groups/8829a4e6f66346ba93b72b9801c5b9e7/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:12.933273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8e317-a186-45e5-96a3-5534410709a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d23d643906b4c84bfbe35da532a7c31, group_id=8829a4e6f66346ba93b72b9801c5b9e7, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:12.934722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8e317-a186-45e5-96a3-5534410709a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:12.956775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 207/415] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Sep 13 13:59:12 2022] PUT /identity/v3/projects/4d23d643906b4c84bfbe35da532a7c31/groups/8829a4e6f66346ba93b72b9801c5b9e7/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:13.270146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddcc3b-378f-4d9d-8680-e31c07da47a0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:13.270544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddcc3b-378f-4d9d-8680-e31c07da47a0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:13.270726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddcc3b-378f-4d9d-8680-e31c07da47a0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:13.272278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 209/416] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:13.277989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d716f-3390-431a-a5b0-035a60714a2b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:13.278273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d716f-3390-431a-a5b0-035a60714a2b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:13.278547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d716f-3390-431a-a5b0-035a60714a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:13.294949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:13.306817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-790d716f-3390-431a-a5b0-035a60714a2b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:13.308035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-790d716f-3390-431a-a5b0-035a60714a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:13.387106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 208/417] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:13.720242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddcc3b-378f-4d9d-8680-e31c07da47a0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:13.783921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:13.793906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf9eb2-a3ec-4278-8688-58bac42cd99f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:13.794421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf9eb2-a3ec-4278-8688-58bac42cd99f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:13.794421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf9eb2-a3ec-4278-8688-58bac42cd99f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:13.797534 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 209/418] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:13.806627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b137-19eb-4d64-99bb-2418a802ca12 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:13.806905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b137-19eb-4d64-99bb-2418a802ca12 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:13.807178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b137-19eb-4d64-99bb-2418a802ca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:13.835810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e128b137-19eb-4d64-99bb-2418a802ca12 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:13.837076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e128b137-19eb-4d64-99bb-2418a802ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:13.844810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddcc3b-378f-4d9d-8680-e31c07da47a0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:13.897334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666d51-4f1e-4101-a15d-68aa0a3bf4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:13.898447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666d51-4f1e-4101-a15d-68aa0a3bf4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:13.899046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666d51-4f1e-4101-a15d-68aa0a3bf4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:13.899539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666d51-4f1e-4101-a15d-68aa0a3bf4a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:13.915593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 210/419] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:13.916989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc7c99-3e40-4aea-9de4-0b822405a0a4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:13.917290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc7c99-3e40-4aea-9de4-0b822405a0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:13.917561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc7c99-3e40-4aea-9de4-0b822405a0a4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:13.920734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af666d51-4f1e-4101-a15d-68aa0a3bf4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:13.926216 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 211/420] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-af666d51-4f1e-4101-a15d-68aa0a3bf4a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:13.926706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:13.927590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:13.928376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:13.928977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 210/421] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:13.933837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc7c99-3e40-4aea-9de4-0b822405a0a4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:13.935789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8404-61cb-429a-8ba3-8485c50c523e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:13.936184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8404-61cb-429a-8ba3-8485c50c523e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:13.937027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb8404-61cb-429a-8ba3-8485c50c523e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:13.967074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e2cb8404-61cb-429a-8ba3-8485c50c523e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:13.968799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e2cb8404-61cb-429a-8ba3-8485c50c523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:14.074157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:14.107918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 211/422] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:14.121364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc7c99-3e40-4aea-9de4-0b822405a0a4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:14.160797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:14.161401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:14.161676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:14.161981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:14.162708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:14.163007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:14.164440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:14.174077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 212/423] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:14.179450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb8404-61cb-429a-8ba3-8485c50c523e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:14.207077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:14.233283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:14.351484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb8404-61cb-429a-8ba3-8485c50c523e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:14.352217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6582ff-f83d-4f2a-b5cd-76a98080a5f6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:14.379139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:14.395393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:14.396076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:14.396353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:14.396672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:14.406618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:14.407631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:14.408108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:14.408574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:14.419021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:14.425760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:14.426686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:14.427499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 213/424] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:59:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:14.432091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:14.542657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:14.544123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:14.563600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ce2f-9106-436d-b7b3-1348a7f1576a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:14.583270 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:14.603219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:14.603751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:14.604029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:14.604301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:14.605079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:14.605363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:14.606887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:14.615265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 214/425] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:14.620698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:14.661816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 212/426] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:14.663046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:14.682979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:14.753157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d9aa1-2a91-4c3d-b9a8-d7d77c1a19bf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:14.771349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a636b-6f47-4946-9cfb-9603a9312d6d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:14.799138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:14.800007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:14.800329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:14.800640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:14.810108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:14.810971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:14.812002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:14.812474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:14.814135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:14.815859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:14.823121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:14.824094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 13:59:14.824899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:14.829629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 215/427] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:14.830584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:14.895402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:14.897198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:14.974006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914074b-df4d-49bf-bd71-4300dfe66bc3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:14.988912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.008901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 213/428] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:15.010175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.026926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.027818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.028097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.028811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.030285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.031752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.034063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 216/429] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:14 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.034833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.059408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.148272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3048f2-913f-44ed-803a-59747dcc59cf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.160858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffb5dc-bafd-4e62-9220-00411ce70a0c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.185627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.186794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.187566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.188046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.188922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:15.189427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.191013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.200793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 214/430] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.201950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.209313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.209987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.210271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.210548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] PATH_INFO: `/v3/projects/18b7b138b42148d79097c7b2e243b955` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.211754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] RBAC: Authorizing `identity:get_project(project_id=18b7b138b42148d79097c7b2e243b955)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.213212 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.217970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 217/431] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/projects/18b7b138b42148d79097c7b2e243b955 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.219264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.284142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.332697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.343268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921df65-a8d1-4f75-9788-897a9803bac1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.354263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3dc96-5597-4620-a68d-1fab984aad01 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.383816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.385128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.385707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.386293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.387578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.389187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.389666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01c25d-b5e3-497a-9fea-cf74380662a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.390370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c25d-b5e3-497a-9fea-cf74380662a6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.390644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c25d-b5e3-497a-9fea-cf74380662a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.390920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01c25d-b5e3-497a-9fea-cf74380662a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.391939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 215/432] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.392924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.467279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01c25d-b5e3-497a-9fea-cf74380662a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.469652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01c25d-b5e3-497a-9fea-cf74380662a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.510714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7308eae-5ee1-47fc-8442-8740f4ed5bed None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.518529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.551084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954e83d-6b33-4329-b8e1-02f25de21f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.551720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954e83d-6b33-4329-b8e1-02f25de21f03 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.552001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954e83d-6b33-4329-b8e1-02f25de21f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.552277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954e83d-6b33-4329-b8e1-02f25de21f03 None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.553387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954e83d-6b33-4329-b8e1-02f25de21f03 None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.554852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954e83d-6b33-4329-b8e1-02f25de21f03 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.556413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 216/433] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.557531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085f019-2086-4434-a3a5-59af2a7ee5f1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.557836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085f019-2086-4434-a3a5-59af2a7ee5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.558115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085f019-2086-4434-a3a5-59af2a7ee5f1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.559292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 217/434] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:15.561828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085f019-2086-4434-a3a5-59af2a7ee5f1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.577686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 218/435] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:15.579026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f1169-7742-429f-981e-b565e1d56690 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.579320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f1169-7742-429f-981e-b565e1d56690 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.579595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f1169-7742-429f-981e-b565e1d56690 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.584661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 219/436] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:15.584661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f1169-7742-429f-981e-b565e1d56690 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.594957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.667164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085f019-2086-4434-a3a5-59af2a7ee5f1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.680586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f1169-7742-429f-981e-b565e1d56690 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.700733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6624a-d7f7-473a-bfc0-d685fdecaf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.701332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6624a-d7f7-473a-bfc0-d685fdecaf88 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.701606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6624a-d7f7-473a-bfc0-d685fdecaf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.701906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6624a-d7f7-473a-bfc0-d685fdecaf88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.703347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6624a-d7f7-473a-bfc0-d685fdecaf88 None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=18b7b138b42148d79097c7b2e243b955) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:15.703651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6624a-d7f7-473a-bfc0-d685fdecaf88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.705408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6624a-d7f7-473a-bfc0-d685fdecaf88 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.711202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 218/437] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.712396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e93c7c-137e-454f-be00-9e61d9878bc2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.712679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e93c7c-137e-454f-be00-9e61d9878bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.712951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e93c7c-137e-454f-be00-9e61d9878bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.713799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.714403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.714678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.714948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.715681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=18b7b138b42148d79097c7b2e243b955) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:15.715990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.717526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.724258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 220/438] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=18b7b138b42148d79097c7b2e243b955&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.725132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:15.739932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f0e93c7c-137e-454f-be00-9e61d9878bc2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:15.741587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f0e93c7c-137e-454f-be00-9e61d9878bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:15.754164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.858928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 219/439] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:15.862235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508ebce-6db6-46d2-b553-0597866a07c6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.862235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508ebce-6db6-46d2-b553-0597866a07c6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.862235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508ebce-6db6-46d2-b553-0597866a07c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.885114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe748d9-9e04-4a01-bbce-55e0890f1c6e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:15.899611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c508ebce-6db6-46d2-b553-0597866a07c6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:15.902028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c508ebce-6db6-46d2-b553-0597866a07c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:15.935454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:15.938317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:15.939081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:15.939341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:15.940245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:15.941414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:15.942990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:15.944737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 221/440] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:15.950296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.001344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 220/441] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:16.048766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:16.072939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0df85c-b2cf-4f40-9faa-d6b1b512ab8f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.108122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.109403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.109757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.110073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.111193 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:16.112631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:16.114587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 222/442] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:15 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:16.120432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508ebce-6db6-46d2-b553-0597866a07c6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.168342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.239790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508ebce-6db6-46d2-b553-0597866a07c6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.294007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42c15a-3d90-4ef0-a6d5-019367814d77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.297829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.298786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.299277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.299752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.301045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=6bc93361bf6e4aeb91fd6c5713557177, scope.project.id=883fcb9f0c0c404586e1b021177d762e) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:16.301570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:16.303939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:16.315361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 221/443] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=6bc93361bf6e4aeb91fd6c5713557177 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:16.336392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.337045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.337321 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.337594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.354930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.355486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:16.361242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:16.362229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:16.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 223/444] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:16.370707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.384317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:16.391135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:16.501636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf69d3-ab69-46d7-8384-d2d8ac893ebe None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.523055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401fb0-bf22-41a6-8c99-0aa3990a4f13 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.555576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.556110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.556391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.556650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.557321 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:16.557622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:16.559099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:16.568981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 224/445] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:16.570550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.571194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.571474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.571754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.575780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.584099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:16.589542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:16.590460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:16.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 222/446] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:16.596646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.654165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:16.691740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bb063-62c8-4da3-bc84-ae330c9e40e0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.692416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:16.717321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e047f99-04b6-4b8c-a0b5-8ae95adf49da None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.731761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.732366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.732640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.732971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.757911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.758524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.758801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.759070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.759729 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:16.760027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:16.761380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:16.770108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 223/447] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:16.775720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.803220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:16.804655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:16.869040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:16.873350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03653e05-994b-4cdb-8664-7986787aede4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:16.904209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 225/448] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 13:59:16.907302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:16.907904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:16.908182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:16.908453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:16.909757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:16.920773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:16.976028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:16.977465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:17.006045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab64eb-e508-4b8a-9b23-2733249dcc9a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:17.039726 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:17.040335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:17.040611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:17.040886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] PATH_INFO: `/v3/projects/883fcb9f0c0c404586e1b021177d762e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:17.042012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] RBAC: Authorizing `identity:get_project(project_id=883fcb9f0c0c404586e1b021177d762e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:17.043441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:17.045030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 226/449] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/projects/883fcb9f0c0c404586e1b021177d762e => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:17.050932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:17.075307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 224/450] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 13:59:17.080616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:17.084571 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:17.123389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:17.148608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a59421-8814-4c1e-a609-1bff435b2d3f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:17.178416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea02e6-b3c8-4754-abb9-61edeb842933 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:17.182463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:17.183079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:17.183355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:17.183622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] PATH_INFO: `/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:17.184733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] RBAC: Authorizing `identity:get_group(group_id=762554bb5ca74134b58d9b5a3dd2a5b5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:17.186198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:17.188288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 227/451] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:17 2022] GET /identity/v3/groups/762554bb5ca74134b58d9b5a3dd2a5b5 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:17.194128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:17.211801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:17.212427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:17.212701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:17.212971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] PATH_INFO: `/v3/projects/4d23d643906b4c84bfbe35da532a7c31` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:17.214111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] RBAC: Authorizing `identity:get_project(project_id=4d23d643906b4c84bfbe35da532a7c31)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:17.215528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:17.217257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 225/452] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 13:59:17 2022] GET /identity/v3/projects/4d23d643906b4c84bfbe35da532a7c31 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:17.222419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:17.290712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c4cb-eee1-4ca7-b1ed-a280f799cd13 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:17.292988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:17.324046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30e1a0-187d-436e-bf1a-5407c86adf85 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:17.325515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:17.326142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:17.326423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:17.326686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:17.327427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] RBAC: Adding query filter params (group.id=762554bb5ca74134b58d9b5a3dd2a5b5, role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=883fcb9f0c0c404586e1b021177d762e) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:17.327728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:17.329162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:17.334801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 228/453] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:17 2022] GET /identity/v3/role_assignments?group.id=762554bb5ca74134b58d9b5a3dd2a5b5&scope.project.id=883fcb9f0c0c404586e1b021177d762e&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:17.337095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:17.368902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a193c-2bfe-4148-9eda-822b44340549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:17.369553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a193c-2bfe-4148-9eda-822b44340549 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:17.369883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a193c-2bfe-4148-9eda-822b44340549 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:17.370158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a193c-2bfe-4148-9eda-822b44340549 None admin] PATH_INFO: `/v3/groups/8829a4e6f66346ba93b72b9801c5b9e7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:17.371348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a193c-2bfe-4148-9eda-822b44340549 None admin] RBAC: Authorizing `identity:get_group(group_id=8829a4e6f66346ba93b72b9801c5b9e7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:17.372804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a193c-2bfe-4148-9eda-822b44340549 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:17.375238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 226/454] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Sep 13 13:59:17 2022] GET /identity/v3/groups/8829a4e6f66346ba93b72b9801c5b9e7 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:17.381135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:17.488112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2d73d-fcbd-4156-8694-b1812c8f606d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:17.528559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e622f-3946-4819-ac0d-792ff1848ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:17.529175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e622f-3946-4819-ac0d-792ff1848ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:17.529457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e622f-3946-4819-ac0d-792ff1848ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:17.529801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e622f-3946-4819-ac0d-792ff1848ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:17.530550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e622f-3946-4819-ac0d-792ff1848ab8 None admin] RBAC: Adding query filter params (group.id=8829a4e6f66346ba93b72b9801c5b9e7, role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=4d23d643906b4c84bfbe35da532a7c31) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:17.530934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e622f-3946-4819-ac0d-792ff1848ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:17.532381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e622f-3946-4819-ac0d-792ff1848ab8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:17.538829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 229/455] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Sep 13 13:59:17 2022] GET /identity/v3/role_assignments?group.id=8829a4e6f66346ba93b72b9801c5b9e7&scope.project.id=4d23d643906b4c84bfbe35da532a7c31&role.id=96af379f1477489b8542698d3c012f1a => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:20.365281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185597d0-ea4c-4960-b045-4a3b32d54508 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:20.366336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185597d0-ea4c-4960-b045-4a3b32d54508 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:20.366834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185597d0-ea4c-4960-b045-4a3b32d54508 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:20.368612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 227/456] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:20.374052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f465-370e-4599-b70d-f09abee288c7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:20.374585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f465-370e-4599-b70d-f09abee288c7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:20.374662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f465-370e-4599-b70d-f09abee288c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:20.404452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b138f465-370e-4599-b70d-f09abee288c7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:20.406355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b138f465-370e-4599-b70d-f09abee288c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:20.444013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:20.456070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0175-27e5-4337-b451-e5e375d33fb9 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:20.456656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0175-27e5-4337-b451-e5e375d33fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:20.457226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e0175-27e5-4337-b451-e5e375d33fb9 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:20.459003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 228/457] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:20.464447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960259d-acfc-4cd0-a9e0-a165fdbc8ac2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:20.464968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960259d-acfc-4cd0-a9e0-a165fdbc8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:20.465461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960259d-acfc-4cd0-a9e0-a165fdbc8ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:20.497193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8960259d-acfc-4cd0-a9e0-a165fdbc8ac2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:20.498416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8960259d-acfc-4cd0-a9e0-a165fdbc8ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:20.512115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 230/458] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:20.513579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29fb13-1374-4f5f-b4a6-150a5758ada9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:20.514059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29fb13-1374-4f5f-b4a6-150a5758ada9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:20.514496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29fb13-1374-4f5f-b4a6-150a5758ada9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:20.516729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 231/459] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:20.523087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9029ec-4fa0-4e72-b38f-6b6839677bce None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:20.523635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9029ec-4fa0-4e72-b38f-6b6839677bce None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:20.524124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9029ec-4fa0-4e72-b38f-6b6839677bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:20.558224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8f9029ec-4fa0-4e72-b38f-6b6839677bce None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:20.559409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f9029ec-4fa0-4e72-b38f-6b6839677bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:20.640984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 229/460] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:20.689323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 232/461] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:20.787041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960259d-acfc-4cd0-a9e0-a165fdbc8ac2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:20.932549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960259d-acfc-4cd0-a9e0-a165fdbc8ac2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:20.945208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:20.968528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9029ec-4fa0-4e72-b38f-6b6839677bce None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:20.986309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:20.988061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:20.988639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:20.990497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.008901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.010607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:21.040434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:21.041591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:21.042702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:21.042702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:21.042702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:21.047911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 230/462] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:21.056761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.131339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:21.143853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9029ec-4fa0-4e72-b38f-6b6839677bce None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.182180 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304ad91-1571-42f2-bb81-f2c73a1df784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.182378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e6aed-0747-41b9-875a-7733bc53b200 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.182859 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304ad91-1571-42f2-bb81-f2c73a1df784 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.183091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304ad91-1571-42f2-bb81-f2c73a1df784 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.183366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304ad91-1571-42f2-bb81-f2c73a1df784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.194038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304ad91-1571-42f2-bb81-f2c73a1df784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.195508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304ad91-1571-42f2-bb81-f2c73a1df784 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8304ad91-1571-42f2-bb81-f2c73a1df784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:21.212090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:21.213091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:21.213993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:21.213993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:21.213993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:21.213993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 233/463] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:21.214669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0cf02-ea33-4b74-8e63-0febeb9c0223 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.214776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0cf02-ea33-4b74-8e63-0febeb9c0223 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.214998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0cf02-ea33-4b74-8e63-0febeb9c0223 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.216218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 234/464] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:21.216910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0cf02-ea33-4b74-8e63-0febeb9c0223 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.226383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.226995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.227379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.227659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.228306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:21.228611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.230023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.242866 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 231/465] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:21.243767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.270877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:21.335562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0cf02-ea33-4b74-8e63-0febeb9c0223 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.358111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d2c23-50c5-46a3-b8fc-f8633e3984c8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.370709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca3b86-eeaa-4c3a-8cb0-d09a8a4ea4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.371359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca3b86-eeaa-4c3a-8cb0-d09a8a4ea4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.371636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca3b86-eeaa-4c3a-8cb0-d09a8a4ea4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.371899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca3b86-eeaa-4c3a-8cb0-d09a8a4ea4e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.383625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca3b86-eeaa-4c3a-8cb0-d09a8a4ea4e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.385192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca3b86-eeaa-4c3a-8cb0-d09a8a4ea4e8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.388003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:21.398643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.399279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.399548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.399824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.400471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:21.400777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b5ca3b86-eeaa-4c3a-8cb0-d09a8a4ea4e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:21.400872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:21.401713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:21.402864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:21.402864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:21.402864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:21.403024 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.405071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 235/466] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:21.406670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb9e7a-6532-4b93-be5d-126ce88ee3de None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.406942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb9e7a-6532-4b93-be5d-126ce88ee3de None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.407207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb9e7a-6532-4b93-be5d-126ce88ee3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.425289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 232/467] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:21.425289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.429815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4beb9e7a-6532-4b93-be5d-126ce88ee3de None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:21.431002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4beb9e7a-6532-4b93-be5d-126ce88ee3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:21.514469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:21.523538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 236/468] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:21.524736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb9e7a-6532-4b93-be5d-126ce88ee3de None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.544526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293aa6-d34b-4c51-b994-e740bd80b005 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.590064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.590977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.591430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.591880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.593270 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.594777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.619269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 233/469] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:59:21 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:21.620634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.646522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb9e7a-6532-4b93-be5d-126ce88ee3de None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.677626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:21.688540 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809911d-db82-445e-85ed-55eff0fbadd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.689413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809911d-db82-445e-85ed-55eff0fbadd5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.689869 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809911d-db82-445e-85ed-55eff0fbadd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.690281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809911d-db82-445e-85ed-55eff0fbadd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.691138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809911d-db82-445e-85ed-55eff0fbadd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:21.691651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809911d-db82-445e-85ed-55eff0fbadd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.693262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809911d-db82-445e-85ed-55eff0fbadd5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.706424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 237/470] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:21.707948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c9c5-e21b-4421-8d84-17a3551ab61b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.708373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c9c5-e21b-4421-8d84-17a3551ab61b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.708854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c9c5-e21b-4421-8d84-17a3551ab61b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.714754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 238/471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:21.715884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609c9c5-e21b-4421-8d84-17a3551ab61b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.774218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b00a2-2df8-4dd1-a918-dfab9ef213b5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.821174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef350147-044a-4e57-a8c9-c12e1200cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.821838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350147-044a-4e57-a8c9-c12e1200cdbd None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.822118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350147-044a-4e57-a8c9-c12e1200cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.822388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef350147-044a-4e57-a8c9-c12e1200cdbd None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.823036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef350147-044a-4e57-a8c9-c12e1200cdbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.824528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef350147-044a-4e57-a8c9-c12e1200cdbd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.832149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609c9c5-e21b-4421-8d84-17a3551ab61b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:21.835218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:21.850172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 234/472] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:59:21 2022] POST /identity/v3/users => generated 246 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:21.851538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269bca-53db-4659-8342-a28f0eadbb7c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.851827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269bca-53db-4659-8342-a28f0eadbb7c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.852200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55269bca-53db-4659-8342-a28f0eadbb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.872656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-55269bca-53db-4659-8342-a28f0eadbb7c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:21.873968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-55269bca-53db-4659-8342-a28f0eadbb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:21.881028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:21.881667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:21.882001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:21.882275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:21.882980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:21.885229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:21.920330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 239/473] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:59:21 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:21.922033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:21.982857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 235/474] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:22.004000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:22.076458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60b16b-5bd2-421f-ab2f-00ea08ca7b70 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:22.113578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:22.114261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:22.114538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:22.114820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:22.128154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:22.129652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:22.146080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:22.147066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:22.147977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:22.147977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:22.147977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:22.147977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 240/475] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:22.153779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55269bca-53db-4659-8342-a28f0eadbb7c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:22.261996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55269bca-53db-4659-8342-a28f0eadbb7c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:22.287543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:22.288294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:22.303105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:22.303718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:22.303991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:22.304261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:22.304915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:22.305214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:22.306621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:22.319595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 236/476] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:22.328702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:22.441242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:22.454873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187aacf-7c80-4b4e-8277-b6020266acb5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:22.507115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3ef4c-a24c-48ca-b305-1e2a79b59d50 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:22.528434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:22.529065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:22.529349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:22.529622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:22.530398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:22.531935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:22.548223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.sql.core [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 13:59:22.548223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 13:59:22.548223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [parameters: {'id': '25f2ebc2d70043849c6e8c717b7e4f90', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 13:59:22.548223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81605) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 13 13:59:22.565163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 13:59:22.566121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application yield resource Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise self.value Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application session.commit() Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 13:59:22.567053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.flush() Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application compat.raise_( Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 13:59:22.567912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 13:59:22.568787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 13:59:22.569637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [parameters: {'id': '25f2ebc2d70043849c6e8c717b7e4f90', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:22.570449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 13:59:22.573568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:22.581100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 241/477] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 13 13:59:22 2022] POST /identity/v3/roles => generated 144 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:22.582782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:22.603465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:22.604094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:22.604383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:22.604660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:22.605319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:22.606838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:22.649342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 237/478] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:59:22 2022] POST /identity/v3/users => generated 249 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:22.754450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceebacb-03e4-4cbd-ba52-dc0776c565b3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:22.797066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139a46b-debd-4a8b-8310-d214b9079fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:22.798112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139a46b-debd-4a8b-8310-d214b9079fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:22.798592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139a46b-debd-4a8b-8310-d214b9079fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:22.799118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139a46b-debd-4a8b-8310-d214b9079fb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:22.810511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139a46b-debd-4a8b-8310-d214b9079fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a139a46b-debd-4a8b-8310-d214b9079fb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:22.816561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:22.817530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 13:59:22.818375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:22.821146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 242/479] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 13:59:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:22.828043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:22.889906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:22.980447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00def6-af0b-4578-b3ec-cf0d9aced574 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:23.018281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbd19a-d4b0-479c-a947-a08ce2a4688e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:23.019133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbd19a-d4b0-479c-a947-a08ce2a4688e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:23.019568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbd19a-d4b0-479c-a947-a08ce2a4688e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:23.020014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbd19a-d4b0-479c-a947-a08ce2a4688e None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:23.020831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbd19a-d4b0-479c-a947-a08ce2a4688e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:23.021282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbd19a-d4b0-479c-a947-a08ce2a4688e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:23.022820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbd19a-d4b0-479c-a947-a08ce2a4688e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:23.033301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 238/480] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:24.420278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6119fce-6691-42af-b617-3aba1293ef38 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:24.420905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6119fce-6691-42af-b617-3aba1293ef38 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:24.420905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6119fce-6691-42af-b617-3aba1293ef38 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:24.422659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 243/481] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:24.428373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45476652-c37a-48a2-ac52-b413ae661499 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:24.428895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45476652-c37a-48a2-ac52-b413ae661499 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:24.428895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45476652-c37a-48a2-ac52-b413ae661499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:24.458959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-45476652-c37a-48a2-ac52-b413ae661499 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:24.460174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-45476652-c37a-48a2-ac52-b413ae661499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:24.560365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 239/482] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:24.602696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a320ae8-7d8c-4125-9a72-9a113de0308e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:24.602999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a320ae8-7d8c-4125-9a72-9a113de0308e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:24.603239 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a320ae8-7d8c-4125-9a72-9a113de0308e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:24.604554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 244/483] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:24.609928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678fa9a-95f0-4fa5-a0f6-b7df4d8213a5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:24.610213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678fa9a-95f0-4fa5-a0f6-b7df4d8213a5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:24.610480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678fa9a-95f0-4fa5-a0f6-b7df4d8213a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:24.638666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2678fa9a-95f0-4fa5-a0f6-b7df4d8213a5 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:24.639935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2678fa9a-95f0-4fa5-a0f6-b7df4d8213a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:24.672504 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:24.685665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4d68-5118-4942-ab72-19b16bccaa44 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:24.685983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4d68-5118-4942-ab72-19b16bccaa44 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:24.686256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4d68-5118-4942-ab72-19b16bccaa44 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:24.688503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 245/484] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:24.697270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b989-f632-4c7c-bd97-cfa64002511b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:24.697550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b989-f632-4c7c-bd97-cfa64002511b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:24.697842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b989-f632-4c7c-bd97-cfa64002511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:24.727485 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 240/485] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:24.734863 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5eb0b989-f632-4c7c-bd97-cfa64002511b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:24.736043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5eb0b989-f632-4c7c-bd97-cfa64002511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:24.759408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:24.837683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 246/486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:24.949343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678fa9a-95f0-4fa5-a0f6-b7df4d8213a5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.051592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b989-f632-4c7c-bd97-cfa64002511b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.089606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678fa9a-95f0-4fa5-a0f6-b7df4d8213a5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.147712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.148351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.148626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.148890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.169369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:25.175412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:25.176298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:25.177100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:25.185932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 241/487] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:25.187346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.222910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b989-f632-4c7c-bd97-cfa64002511b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.232806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:25.280634 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.281279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.281570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.282043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.296957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:25.302272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:25.303177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:25.304000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 247/488] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:25.304673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.320796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:25.350050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb617dc-7e21-47ea-bbdb-27e935b39c01 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.398439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.400066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.400453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.400831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.402097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:25.402774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:25.405031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:25.416333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 242/489] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:25.417203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.439510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0f8be-1e5d-4d05-8ffd-a356fa24673e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.478339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.479076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.479452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.479826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.494632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:25.502474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:25.503562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:25.504547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:25.507776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 248/490] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:25.508780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.528658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:25.554967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396ce0-d0e0-4db7-b205-ab7f11da9c26 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.591584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520af553-3a55-4344-b758-15d60adb843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.592497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520af553-3a55-4344-b758-15d60adb843c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.593556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520af553-3a55-4344-b758-15d60adb843c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.594121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520af553-3a55-4344-b758-15d60adb843c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.594976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520af553-3a55-4344-b758-15d60adb843c None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:25.595764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520af553-3a55-4344-b758-15d60adb843c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:25.596972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:25.599384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520af553-3a55-4344-b758-15d60adb843c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:25.608868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 243/491] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:25.610019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520af553-3a55-4344-b758-15d60adb843c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.635463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfe87f-c734-4304-847c-62f8025cfe0f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.670520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb78419-e076-4087-82a1-836252ca1620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.671795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb78419-e076-4087-82a1-836252ca1620 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.672116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb78419-e076-4087-82a1-836252ca1620 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.672338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb78419-e076-4087-82a1-836252ca1620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.682999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb78419-e076-4087-82a1-836252ca1620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:25.684448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb78419-e076-4087-82a1-836252ca1620 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-edb78419-e076-4087-82a1-836252ca1620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:25.698691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:25.699680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:25.700627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:25.700627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:25.700627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:25.700627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 249/492] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:25.701264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865676a1-28ee-4456-848a-e86b61dfc40b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.701264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865676a1-28ee-4456-848a-e86b61dfc40b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.701442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865676a1-28ee-4456-848a-e86b61dfc40b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.707387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 250/493] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:25.707387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865676a1-28ee-4456-848a-e86b61dfc40b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.728183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520af553-3a55-4344-b758-15d60adb843c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.764442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.765519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.765839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.766136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.766808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:25.767116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:25.768538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:25.783240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 244/494] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:25.783240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.792346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:25.823768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865676a1-28ee-4456-848a-e86b61dfc40b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.845069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:25.861636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea08fc-a4b4-42e2-94ad-5229986488ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.862304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea08fc-a4b4-42e2-94ad-5229986488ed None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.862582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea08fc-a4b4-42e2-94ad-5229986488ed None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.862855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea08fc-a4b4-42e2-94ad-5229986488ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.874596 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea08fc-a4b4-42e2-94ad-5229986488ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:25.876114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea08fc-a4b4-42e2-94ad-5229986488ed None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-29ea08fc-a4b4-42e2-94ad-5229986488ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:25.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:25.892529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:25.893504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:25.893504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:25.893504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:25.893861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a988bd-e069-4acf-9abc-a3c60aa72fc2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:25.893934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 251/495] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:25.894157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196603ce-cfe6-48ec-abdf-6b8e632e2666 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.894396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196603ce-cfe6-48ec-abdf-6b8e632e2666 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.894633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196603ce-cfe6-48ec-abdf-6b8e632e2666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.913090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-196603ce-cfe6-48ec-abdf-6b8e632e2666 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:25.914300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-196603ce-cfe6-48ec-abdf-6b8e632e2666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:25.929857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:25.930458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.930734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.931006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.931641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:25.931943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:25.933277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:25.945675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 245/496] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:25.946490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:25.982988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 252/497] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:25.984974 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4aef8-d922-45e5-87fa-86ebeab1078d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:25.985257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4aef8-d922-45e5-87fa-86ebeab1078d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:25.985519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4aef8-d922-45e5-87fa-86ebeab1078d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:25.986968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 253/498] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:25.990875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4aef8-d922-45e5-87fa-86ebeab1078d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.034105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.046937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4292d-89aa-414a-9d4b-c79ed269c299 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.082975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.083605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.083882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.084406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.095707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.097143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.098206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4aef8-d922-45e5-87fa-86ebeab1078d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.101232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:26.116570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:26.117558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:26.118435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:26.118435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:26.118435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:26.118435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 246/499] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:26.118658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.141651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c5833-7c0a-4b55-ab8c-131565b2d2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.142303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c5833-7c0a-4b55-ab8c-131565b2d2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.142585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c5833-7c0a-4b55-ab8c-131565b2d2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.142854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c5833-7c0a-4b55-ab8c-131565b2d2d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.143496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c5833-7c0a-4b55-ab8c-131565b2d2d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:26.143800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c5833-7c0a-4b55-ab8c-131565b2d2d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.145213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c5833-7c0a-4b55-ab8c-131565b2d2d6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.157408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 254/500] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:26.158636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb4957-b22d-494f-913c-053bd51898c3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.158922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb4957-b22d-494f-913c-053bd51898c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.159192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb4957-b22d-494f-913c-053bd51898c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.176980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ecbb4957-b22d-494f-913c-053bd51898c3 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:26.178206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ecbb4957-b22d-494f-913c-053bd51898c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:26.233753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb687159-af32-484d-bad5-0cbc804e1c37 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.253047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.256862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 255/501] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:26.257792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb4957-b22d-494f-913c-053bd51898c3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.273070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.273761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.274048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.274359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.282999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.284517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:26.297930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:26.299002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 13:59:26.300218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:26.303571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 247/502] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:26.304360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.357784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.368909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb4957-b22d-494f-913c-053bd51898c3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.404737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.405479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.405781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.406163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.406801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:26.407109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.408479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.415176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2acf7-68f9-4580-b600-601034acf3d4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.421506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 256/503] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:26.426493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.451163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.451793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.452135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.452605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.460543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.462018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:26.473403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:26.474552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 248/504] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:26.475634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.525592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.540120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc64b8-1f56-42ed-a72a-246ee89c4981 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.574029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.574652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.574935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.575204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.581336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a342c-ede1-42f8-9ce6-96d92a41dae6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.585289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:26.590554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:26.591652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 257/505] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:26.592736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.608061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.618571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.619329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.619608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.619877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.620604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:26.620908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.622376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.659764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 249/506] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:26.660613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.693917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd5c9d-eb7d-4cc9-b436-51381bc21fd6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.726444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.727072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.727341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.727610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.734990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.736417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:26.748483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:26.749904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 13:59:26.750826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:26.754094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.754603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 258/507] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:26.755623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.761160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23735a-a977-476a-bbdd-43ad26507a2c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.795326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.795875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.796151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.796420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.797130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:26.797436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.798892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.815804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 250/508] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:26.820371 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.830555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:26.860320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363ac32-b04f-4812-8632-98e2316cf830 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.894122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.894751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.895033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.895305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.907062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.908585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.917542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d07af-13c6-4ee4-b6a3-fc2b5d4abd29 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:26.922961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:26.923864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:26.924712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:26.924712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:26.924712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:26.924712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 259/509] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:26.924926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.951708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:26.952381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:26.952751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:26.953025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:26.953679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:26.954009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:26.955393 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:26.962948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 251/510] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:26.963725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:26.984195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.024610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1789011-0f4f-49fa-948a-f4a737a93b21 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.047918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.059166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.059767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.060041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.060306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.066785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26459ac1-dfa2-4e93-a726-2de98238d1e5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.100644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.101238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.101514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.101815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.102562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:27.102867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.104296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.121238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 252/511] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:27.122046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.129291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.130817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.194967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.220914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5a072-e84a-40c7-ad03-a975c4f1af4a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.229590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 260/512] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:27.230301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.254450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.255045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.255323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.255595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.256196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.323957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.325427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.332241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fef10-ccf7-4691-822f-edbc08ccd270 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.365636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.366262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.366537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.366806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.367435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:27.367731 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.369363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.381269 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 261/513] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:27.382095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.399547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.429976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 253/514] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:27.429976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.472985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.490204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcc11b-9274-407c-ad1c-b6cc20e046e7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.524531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3478f-353c-43f3-8f05-f8423ec5ee81 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.527271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.528160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.528437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.528700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.540675 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.542138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:27.559142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:27.560246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:27.561112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:27.561112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:27.561112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:27.561283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.561405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 262/515] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:27.561483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.561483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.561483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.561671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.604508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.630567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.632003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.662336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb8761-86cb-4546-bd24-d52ae5eaecba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.695252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.696072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.696344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.696616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.699171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.708253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.709672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:27.724182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:27.725179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:27.726077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:27.726077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:27.726077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:27.726077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 263/516] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:27.726077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.738718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 254/517] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:27.738718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.807930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:27.826503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3167da-7168-4d16-a957-68a6f4c5b2b6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.833223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9d9b7-e16e-413f-bd4d-6a81041ab8f2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:27.860683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.861216 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.861492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.861793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.862431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.863857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.866272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:27.866896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:27.867178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:27.867453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:27.878156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:27.879764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:27.885559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 264/518] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:59:27 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:27.886435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:27.895522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:27.896516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:27.897386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:27.897386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:27.897386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:27.897386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 255/519] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:27.897625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:27.949748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.010097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0d859-899c-4e9c-9248-4477bcded8ff None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.017944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04edde-13df-4bda-ab78-7ae7d0722052 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.040731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.051315 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.051928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.052206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.052479 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.053126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:28.053427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.054845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.060527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.061147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.061471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.062365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.063016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:28.063959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.065281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.068197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 256/520] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:28.069127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.080721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 265/521] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:28.081599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.192574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774c216-7445-402f-983b-305161ac7c97 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.193998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f5b2a-3995-4738-b877-bcd52afd13c5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.197081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.229425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.230084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.230362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.230630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.233830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.234605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.235034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.235443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.236222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:28.236656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.238186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.241061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.242528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.251012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 266/522] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:28.252090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:28.262407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:28.263363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:28.264359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:28.264359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:28.264359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:28.264359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 257/523] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:28.265294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.317325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.371585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388a610-1ebd-4ec9-b4b0-d93e2dca50da None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.380787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efe32a-7818-4a37-b730-90ada1683837 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.409203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.409872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.410150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.410416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.417345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.417899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.418630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.418831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.419103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.419461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.427014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.428432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:28.431746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:28.432572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 267/524] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:28.438840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:28.441254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:28.442130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:28.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 258/525] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:28.443639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.466972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.549423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.552476 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df775fc-6fa9-4d12-a223-04b01f782763 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.553453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050611bd-4fdf-4860-89b9-e15939a63c11 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.587618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.588243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.588519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.588784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.591660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.592267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.592554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.592997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.593767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:28.594079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.595459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.595740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.597169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.612332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 259/526] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:28.612332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:28.612910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:28.613934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 268/527] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:28.614939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.703843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.721571 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467210f-2c84-4cc4-865c-1cb57bd224d5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.723636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932864f5-7a15-44ec-b3b2-34837ca3734f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.754842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.755433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.755712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.755980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.756688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:28.756986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.758431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.761187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.761811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.762088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.762355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.763039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:28.763340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.765291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.783848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 260/528] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:28.784737 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.800238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 269/529] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:28.805805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:28.805991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.891369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103afb2-defb-456e-8dbd-e928bc1b3265 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.910390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47ead5-8163-48eb-acb4-dbcba7af2df5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:28.925117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.925779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.926059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.926329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.933334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.934944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.946663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:28.947267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:28.947541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:28.947703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:28.948506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:28.949347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:28.949780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:28.949780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:28.949780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:28.950622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:28.953672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 261/530] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:28.954410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.970442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 270/531] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:28.971531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:28.975631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:29.043850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:29.068546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea67b5-4a6c-4744-8e35-c05fcc900a02 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:29.076681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f70c2e-b59d-4cca-be7e-9c05736c70bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:29.106097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:29.106704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:29.106984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:29.107263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:29.107989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=f7214e6addc646a899efb82283ededc1) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:29.108985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:29.111652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:29.112599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:29.113275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:29.113620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:29.113988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:29.130131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 262/532] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/role_assignments?user.id=f7214e6addc646a899efb82283ededc1&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:29.130131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:29.214965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:29.217566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:29.221957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:29.258206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa132cf-c765-4b93-862c-7f7d069b4919 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:29.303324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:29.305138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:29.305138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:29.305138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:29.305772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:29.307544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:29.309138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:29.324072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 263/533] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:29.325435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:29.333522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:29.370132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 271/534] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:29.371329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:29.443991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be710d91-b946-466c-a727-1033cc72de6b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:29.481901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:29.482460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:29.482745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:29.483014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:29.484557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=6a4ea803f6e44648bb217abd00e11c3c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:29.484872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:29.486327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:29.491580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7a56-7d9b-4773-9030-8b294006f801 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:29.492523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 264/535] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:29 2022] GET /identity/v3/role_assignments?user.id=6a4ea803f6e44648bb217abd00e11c3c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:29.493341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:29.494001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:29.533771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:29.534840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:29.535326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:29.536451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:29.537474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=c95553acbf23423993a4d25e83ecee48) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:29.538028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:29.539735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:29.547184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 272/536] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:29 2022] GET /identity/v3/role_assignments?user.id=c95553acbf23423993a4d25e83ecee48&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:29.569900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:29.610247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfed31-5a1f-49fe-8b2d-25c621331500 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:29.648811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:29.649465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:29.649749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:29.650018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:29.662750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:29.664222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:29.681257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:29.682331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:29.683192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:29.683192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:29.683192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:29.683192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 265/537] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:29.690069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:29.840147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8d1cc-0cca-41f2-9c38-ab085fdfaaa1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:29.876321 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:29.876934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:29.877207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:29.877474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:29.878140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:29.878440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:29.879813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:29.892274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 273/538] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:29.897718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:30.024685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160611a8-5eb8-41c1-be08-1aef0363ac58 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:30.042961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:30.062227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9897131-8adc-490c-9253-84d7e6554031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:30.062876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897131-8adc-490c-9253-84d7e6554031 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:30.063158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897131-8adc-490c-9253-84d7e6554031 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:30.063435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9897131-8adc-490c-9253-84d7e6554031 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:30.071372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9897131-8adc-490c-9253-84d7e6554031 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:30.072850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9897131-8adc-490c-9253-84d7e6554031 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a9897131-8adc-490c-9253-84d7e6554031 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:30.086638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:30.087568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:30.088448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 266/539] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:30.095147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:30.147815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:30.222164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073725-26de-415b-8d7d-8649cf423274 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:30.222502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073725-26de-415b-8d7d-8649cf423274 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:30.222727 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3073725-26de-415b-8d7d-8649cf423274 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:30.224203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 267/540] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:59:30.229322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a263e-7d95-4b8d-beb3-d83dbef4bb8f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:30.229611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a263e-7d95-4b8d-beb3-d83dbef4bb8f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:30.229930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a263e-7d95-4b8d-beb3-d83dbef4bb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:30.254539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da930cdb-8460-4a67-b0fd-99138d64ede2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:30.268779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-993a263e-7d95-4b8d-beb3-d83dbef4bb8f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:30.271793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-993a263e-7d95-4b8d-beb3-d83dbef4bb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:30.311007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:30.312154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:30.312647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:30.313125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:30.314051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:30.314572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:30.316191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:30.329775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 274/541] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:30.335860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:30.393228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 268/542] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:30.466207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:30.487312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f4923-9856-42de-a5ef-0643620d4dfa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:30.527946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:30.528548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:30.528824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:30.529093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:30.529834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=11a418a2a35146b59c433421fe507500) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:30.530706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:30.532160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:30.542011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 275/543] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:30 2022] GET /identity/v3/role_assignments?user.id=11a418a2a35146b59c433421fe507500&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:30.682948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a263e-7d95-4b8d-beb3-d83dbef4bb8f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:30.820327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a263e-7d95-4b8d-beb3-d83dbef4bb8f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:30.830687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:30.859391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:30.860128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:30.860412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:30.860689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:30.871444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:30.876854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:30.877758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:30.878589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:30.881836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 269/544] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:30.886808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:31.030611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26762f2-6671-4f3e-91e5-82028545cb01 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:31.065046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:31.065656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:31.065961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:31.066229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:31.066890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:31.067187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:31.068573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:31.078282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 276/545] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:31.084424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:31.227934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd586bb-d55f-41b8-aa64-2bb5beca7065 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:31.270814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:31.271447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:31.271731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:31.272006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:31.289910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:31.291320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:31.318616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:31.319573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:31.320416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:31.320416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:31.320416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:31.320416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 270/546] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:31.326798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:31.355514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:31.465885 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ff2db-7789-449b-be17-a40e0ce06df3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:31.507391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:31.508047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:31.508322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:31.508640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:31.509279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:31.509574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:31.510963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:31.523669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 277/547] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:31.531974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:31.554146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:31.655094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd00eb1-b911-4d5b-a29e-96c5c22a561f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:31.697617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:31.698301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:31.698533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:31.698804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:31.712901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:31.713403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:31.728968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:31.730754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:31.732040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 271/548] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:31.735868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:31.838885 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:31.881589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e4f0d-a841-4c67-9dea-925f649d60b4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:31.933832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504d367-6ec2-4f26-8f2c-54fda197ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:31.934464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d367-6ec2-4f26-8f2c-54fda197ac31 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:31.934721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d367-6ec2-4f26-8f2c-54fda197ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:31.934968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d367-6ec2-4f26-8f2c-54fda197ac31 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:31.935694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504d367-6ec2-4f26-8f2c-54fda197ac31 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:31.935997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504d367-6ec2-4f26-8f2c-54fda197ac31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:31.937994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504d367-6ec2-4f26-8f2c-54fda197ac31 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:31.962129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 278/549] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 13 13:59:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:31.968111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:32.019260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf886b-4f8d-4d2a-876e-69db4b04258b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.019589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf886b-4f8d-4d2a-876e-69db4b04258b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.019817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf886b-4f8d-4d2a-876e-69db4b04258b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.021403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 279/550] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:32.026354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb1c9-0d20-42f3-81ca-15a54165f666 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.026639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb1c9-0d20-42f3-81ca-15a54165f666 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.026914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb1c9-0d20-42f3-81ca-15a54165f666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.052437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9edcb1c9-0d20-42f3-81ca-15a54165f666 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:32.053659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9edcb1c9-0d20-42f3-81ca-15a54165f666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:32.088540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:32.126800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92b4a6-26c5-4820-afbc-31c24c219333 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:32.191087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 280/551] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:32.194508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06d02-0cc7-4062-b2e4-dea842b7ead7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.194508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06d02-0cc7-4062-b2e4-dea842b7ead7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.194508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb06d02-0cc7-4062-b2e4-dea842b7ead7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.194508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 281/552] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:32.197876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:32.198484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.198760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.199030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.204326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a6a2-6d6c-4fb7-b055-90153564fe17 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.204605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a6a2-6d6c-4fb7-b055-90153564fe17 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.204925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a6a2-6d6c-4fb7-b055-90153564fe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.233404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9760a6a2-6d6c-4fb7-b055-90153564fe17 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:32.237012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9760a6a2-6d6c-4fb7-b055-90153564fe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:32.334083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:32.342799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:32.344250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:32.350105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 282/553] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:32.450287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:32.462029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760a6a2-6d6c-4fb7-b055-90153564fe17 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:32.477665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 272/554] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:32.485161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:32.597843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05027e27-2222-4c62-ae54-09c0efc96034 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:32.641756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:32.642410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.642691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.642962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.658362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:32.659848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:32.676783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:32.677791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:32.678684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:32.678684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:32.678684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:32.678684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 273/555] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:32.683967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:32.704707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760a6a2-6d6c-4fb7-b055-90153564fe17 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:32.730079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:32.740749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:32.741387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.741674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.741962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.753009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:32.758160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:32.759131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:32.759964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 283/556] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:32.760663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:32.806911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce2dfb-aea9-44c6-b93c-30c5310945d0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:32.814086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:32.845541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706629c2-1a96-4017-a69b-04338959ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:32.846185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706629c2-1a96-4017-a69b-04338959ef35 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.846465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706629c2-1a96-4017-a69b-04338959ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.846742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706629c2-1a96-4017-a69b-04338959ef35 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.847393 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706629c2-1a96-4017-a69b-04338959ef35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:32.847704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706629c2-1a96-4017-a69b-04338959ef35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:32.849071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706629c2-1a96-4017-a69b-04338959ef35 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:32.865880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 274/557] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:32.867158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e188a4-6c1b-44d4-9055-69b43838052c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.867439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e188a4-6c1b-44d4-9055-69b43838052c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.867710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e188a4-6c1b-44d4-9055-69b43838052c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.868934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 275/558] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:32.869626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e188a4-6c1b-44d4-9055-69b43838052c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:32.873682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7bf5-702e-4537-bda2-c1d5e1be4f7c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:32.910024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:32.910717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:32.911058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:32.911263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:32.922368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:32.927541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:32.929059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 284/559] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:32.929931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:32.982269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:32.993440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e188a4-6c1b-44d4-9055-69b43838052c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.030856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a13364-20c6-4206-a80a-13f505edbad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.031526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a13364-20c6-4206-a80a-13f505edbad4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.031817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a13364-20c6-4206-a80a-13f505edbad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.032090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a13364-20c6-4206-a80a-13f505edbad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.032753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a13364-20c6-4206-a80a-13f505edbad4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:33.033058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a13364-20c6-4206-a80a-13f505edbad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.034447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a13364-20c6-4206-a80a-13f505edbad4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.041605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72df54f-5f4f-4f55-b17a-4b2c3f7e3bd2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.043096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 276/560] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:33.044242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba473c-f257-41d6-b9f3-10aa07c479ca None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.044531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba473c-f257-41d6-b9f3-10aa07c479ca None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.044805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba473c-f257-41d6-b9f3-10aa07c479ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.056945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:33.063895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-88ba473c-f257-41d6-b9f3-10aa07c479ca None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:33.065098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-88ba473c-f257-41d6-b9f3-10aa07c479ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:33.085433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.086213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.086499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.086771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.095649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.097127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:33.111594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:33.112579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:33.113473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 285/561] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:33.114564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.148082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 277/562] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:33.157964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba473c-f257-41d6-b9f3-10aa07c479ca None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.218217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95811f-e3d0-4d88-b81a-7422e6c1e724 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.253387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:33.253544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.253998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.254276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.254544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.255204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:33.255504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.256869 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.265282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 286/563] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:33.266099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.278181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba473c-f257-41d6-b9f3-10aa07c479ca None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.300492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:33.318117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6dfd9-28ad-48c6-9a4f-08571125f16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.318749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6dfd9-28ad-48c6-9a4f-08571125f16a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.319033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6dfd9-28ad-48c6-9a4f-08571125f16a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.319309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6dfd9-28ad-48c6-9a4f-08571125f16a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.330584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6dfd9-28ad-48c6-9a4f-08571125f16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.332035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6dfd9-28ad-48c6-9a4f-08571125f16a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-edd6dfd9-28ad-48c6-9a4f-08571125f16a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:33.348241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:33.349216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:33.350144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:33.350144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:33.350144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:33.350144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 278/564] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:33.350959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805eee-f17f-4d5e-8466-75e13f048d1a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.351241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805eee-f17f-4d5e-8466-75e13f048d1a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.351509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805eee-f17f-4d5e-8466-75e13f048d1a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.356786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 279/565] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:33.356786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805eee-f17f-4d5e-8466-75e13f048d1a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.379716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f5da6-7221-4a09-83fc-0ff9ba5f890b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.415502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.416220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.416444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.416667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.417356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:33.417665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.419143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.431926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 287/566] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:33.432727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.470126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805eee-f17f-4d5e-8466-75e13f048d1a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.503804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b5cac-f2d9-4f8e-b05c-1b349021f7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.504347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b5cac-f2d9-4f8e-b05c-1b349021f7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.504623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b5cac-f2d9-4f8e-b05c-1b349021f7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.504890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b5cac-f2d9-4f8e-b05c-1b349021f7c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.508056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:33.516395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b5cac-f2d9-4f8e-b05c-1b349021f7c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.517860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b5cac-f2d9-4f8e-b05c-1b349021f7c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.526392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:33.534336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e04f61-75fe-49bf-a5bd-b7501f30bd3b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-bb8b5cac-f2d9-4f8e-b05c-1b349021f7c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:33.536047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:33.537166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:33.538105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:33.538105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:33.538105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:33.538105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 280/567] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:33.538635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c4f2-d63d-4879-b30a-2ba42e3542ec None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.538915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c4f2-d63d-4879-b30a-2ba42e3542ec None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.539185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c4f2-d63d-4879-b30a-2ba42e3542ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.557580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bd72c4f2-d63d-4879-b30a-2ba42e3542ec None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:33.558800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bd72c4f2-d63d-4879-b30a-2ba42e3542ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:33.570335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.570937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.571214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.571487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.572119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:33.572418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.573782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.584895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 288/568] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:33.585671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.624269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 281/569] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:33.624973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c4f2-d63d-4879-b30a-2ba42e3542ec None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.688287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2db108-9cdc-4aac-873f-0b297cb313a0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.725143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.725787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.726067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.726336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.727459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c4f2-d63d-4879-b30a-2ba42e3542ec None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.741779 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:33.745795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:33.746344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:33.747234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:33.748116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:33.752671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 289/570] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:33.753906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.764194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.765331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.765608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.765910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.766638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=15e78b0e48a141edad05f3c064ac04f7) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:33.767109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.768354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.771667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:33.785935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 282/571] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/role_assignments?user.id=15e78b0e48a141edad05f3c064ac04f7&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:33.787114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.873778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f192ea0-1d39-4031-8e6e-975c2420ad77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.899350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f106fd3-717b-4e79-afc3-5d80687d1ab1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:33.912530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.913154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.913433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.913706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.914385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:33.914693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.916088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.929546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 290/572] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:33.930458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:33.938386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:33.939029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:33.939306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:33.939578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:33.948231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:33.949748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:33.961973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:33.963168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 13:59:33.964302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:33.967853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 283/573] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:33.968695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.015814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.019002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.048512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684941a0-f19b-411d-897f-ee1e03cd6688 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.082409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31e8a0-a0f9-4a97-b716-5ff828b1d41e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.083887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.084492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.084777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.085045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.085703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:34.086031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.087388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.099979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 291/574] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:34.099979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.123363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.124175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.124631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.124841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.135762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:34.141515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:34.142696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 284/575] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:34.143961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.216766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc6f2d-0c54-42ba-aab4-8b66b7e66e35 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.254191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.254852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.255131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.255404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.257474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.259924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13174c67-ea31-48ba-9e93-8168c6792074 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.261551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.265711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.267948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:34.280040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:34.281441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 292/576] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:34.282506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.312147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.312928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.313304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.313671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.314517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:34.314938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.316414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.338937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 285/577] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:34.338937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.397634 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe96f0-c1d9-4e2b-a7d9-03c227b7d456 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.433680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.434332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.434608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.434875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.446195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.447632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.452337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787e0f4-3be7-420b-b551-86d40b048fd2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:34.461856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:34.463003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:34.463918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:34.463918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:34.463918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:34.463918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 293/578] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:34.464274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.495980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.496677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.496854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.497121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.497806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:34.498208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.499487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.512696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 286/579] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:34.512696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.519607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.540027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.585235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a280c77-9c0d-44d1-ae9f-b8ca2c4b001c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.622592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00211ad6-6584-4106-9e05-abaa4ddb32e3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.623072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.623747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.624617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.624889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.625608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:34.625939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.627384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.649285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 294/580] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:34.649285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.665216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.666062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.666469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.666839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.680474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.681956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:34.697253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:34.698709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:34.699835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:34.699835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:34.699835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:34.699835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 287/581] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:34.699835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.767531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748b88f-297e-471b-a318-92abaff86db0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.769993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.784425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:34.808315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.809005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.809360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.809709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.810448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:34.810700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b65504-d3a2-42d0-b0a4-300aaf4d9863 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.810896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.812654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.829552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 295/582] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:34.829552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.848559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.849301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.849656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.850027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.861379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.862919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:34.877203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:34.878395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:34.879523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:34.879523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:34.879523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:34.884614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 288/583] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:34.884614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:34.941678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8515-ecbf-4c79-996a-eda29e8a810d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:34.976116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:34.976749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:34.977029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:34.977343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:34.989046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:34.990747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:34.990857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f02610-18be-4d9d-ad9c-fe85ef2741c3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:35.004993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:35.006211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:35.007246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:35.007246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:35.007246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:35.007246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 296/584] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:35.007484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.019849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.032352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.032884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.033166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.033411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.033550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.034092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:35.034399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.035969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.049761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 289/585] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:35.049761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.128440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f5b70-e88a-42b3-a52a-69a931c563d5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.160686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c87c5-4d97-4c5a-9ea5-677f53c322c8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.164820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.165447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.165741 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.166024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.174815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.176162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:35.188081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:35.189006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 297/586] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:35.189986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.202125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.203104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.203457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.203935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.204845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:35.205214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.207213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.224655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 290/587] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:35.224655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.273704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.274256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.299008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624a4a-f7b7-443a-abe7-a9cf2333b401 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.333752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.334708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.335220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.335614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.336265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:35.336570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.337155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1363f2-ca76-4c59-9129-9b748db9b336 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.337965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.356754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 298/588] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:35.356754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.376185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.376812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.377090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.377361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.384590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.386067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:35.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:35.399001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:35.399999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 291/589] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:35.400484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.467096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde23cde-3fac-47f2-8d5e-55d72ffd025d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.505244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.506028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.506402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.506801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.507624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:35.508069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.509880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12a37a-2a2f-46af-b58e-23fb3d50e0ab None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.510150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.513478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.520344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.537261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 299/590] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:35.538465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.556666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.557533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.557849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.558132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.566459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.568235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:35.581961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:35.582909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 292/591] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:35.583845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.646705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6bc1d-30c5-45ca-8a23-9959107af490 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.684073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.684870 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.685238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.685617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.693894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.695324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.701475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9f85c-3959-4d6d-9153-4a2d22e2dee7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:35.706864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:35.707944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:35.708900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 300/592] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:35.709404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.746533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.747141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.747423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.747701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.748390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:35.748702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.750438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.768550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 293/593] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:35.768550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.775548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.798931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:35.824735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bf448-9908-42b6-9e42-8e451752318b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.866185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.866841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.867105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.867373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.880050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.881649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.883832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623367e-c2f3-479a-9d58-85de03fcc821 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:35.896084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:35.897055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:35.898691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:35.898691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:35.898691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:35.898691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 301/594] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:35.898691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:35.924081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:35.924689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:35.925323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:35.925323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:35.926007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:35.926345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:35.927835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:35.947989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 294/595] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:35.952773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.009133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b74b19-9d4c-4e4c-8560-ace127aa0151 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.016395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.037709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.044772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.045371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.045820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.046044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.046687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:36.046986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.048403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.052960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e351f-29fd-4658-8ab2-429521203348 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.060145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 302/596] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:36.060941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.087646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.088236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.088515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.088790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/f7214e6addc646a899efb82283ededc1/roles/adad88d8a75747a381a2b923728d27f8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.103305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=f7214e6addc646a899efb82283ededc1, role_id=adad88d8a75747a381a2b923728d27f8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.104737 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.117517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 295/597] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:59:35 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/f7214e6addc646a899efb82283ededc1/roles/adad88d8a75747a381a2b923728d27f8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:36.118427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.171312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ed1d3-02aa-4e40-9ca6-b823dd919d7f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.212655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b361384-a99e-41b6-9d58-5f56a08b37dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.213480 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b361384-a99e-41b6-9d58-5f56a08b37dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.213899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b361384-a99e-41b6-9d58-5f56a08b37dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.214279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b361384-a99e-41b6-9d58-5f56a08b37dc None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.215056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b361384-a99e-41b6-9d58-5f56a08b37dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:36.215494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b361384-a99e-41b6-9d58-5f56a08b37dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.216984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b361384-a99e-41b6-9d58-5f56a08b37dc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.229884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 303/598] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:36.231696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fee06-8181-44de-8ee7-2562a5fccbfa None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.232126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fee06-8181-44de-8ee7-2562a5fccbfa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.232577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fee06-8181-44de-8ee7-2562a5fccbfa None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.234772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 304/599] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:36.235767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fee06-8181-44de-8ee7-2562a5fccbfa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.250601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e169b-d1db-4042-8bde-2cb991da44cd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.268502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.298695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.299502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.299888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.300266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.303930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.313929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.315804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:36.330830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:36.331885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:36.332794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:36.332794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:36.332794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:36.332794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 296/600] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:36.332794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.357051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fee06-8181-44de-8ee7-2562a5fccbfa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.391699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aaa1af-2b4c-439a-a3d9-295fae28e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.392641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaa1af-2b4c-439a-a3d9-295fae28e90c None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.392919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaa1af-2b4c-439a-a3d9-295fae28e90c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.393194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aaa1af-2b4c-439a-a3d9-295fae28e90c None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/6a4ea803f6e44648bb217abd00e11c3c/roles/adad88d8a75747a381a2b923728d27f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.408941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaa1af-2b4c-439a-a3d9-295fae28e90c None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=6a4ea803f6e44648bb217abd00e11c3c, role_id=adad88d8a75747a381a2b923728d27f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.410422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aaa1af-2b4c-439a-a3d9-295fae28e90c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.429489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 305/601] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:59:36 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/6a4ea803f6e44648bb217abd00e11c3c/roles/adad88d8a75747a381a2b923728d27f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:36.429489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9a678-c0aa-4291-a409-1044d2ccbdb7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.429489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9a678-c0aa-4291-a409-1044d2ccbdb7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.429489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9a678-c0aa-4291-a409-1044d2ccbdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.450222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19e41b-d86d-4b83-be7e-64fa44b96783 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.452498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bab9a678-c0aa-4291-a409-1044d2ccbdb7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:36.454810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bab9a678-c0aa-4291-a409-1044d2ccbdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:36.493612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.494315 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.494595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.494867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.503738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.505203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:36.522666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:36.523837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:36.524960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 297/602] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:36.525475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.559370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 306/603] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:36.560548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9a678-c0aa-4291-a409-1044d2ccbdb7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.570540 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.604051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.690325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9a678-c0aa-4291-a409-1044d2ccbdb7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.699558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac3ad-eb2a-4d0a-9384-9ac5eea4d48f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.748223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.748223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.748223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.748223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.748686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:36.748686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.748686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.751310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.751927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.752207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.752478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.753120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:36.753421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.754831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.762478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 307/604] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:36.768796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.785173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 298/605] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:36.791469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:36.888721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.933128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8be80-d5c6-4d0b-ae17-4d1233c295ed None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.933356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca88c01-591b-4b20-bd5c-89b23ffaba13 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:36.941603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:36.972239 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.972910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.973190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.973589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/c95553acbf23423993a4d25e83ecee48/roles/adad88d8a75747a381a2b923728d27f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.974265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:36.975134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:36.975419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:36.975695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:36.982885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.984871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.990150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=c95553acbf23423993a4d25e83ecee48, role_id=adad88d8a75747a381a2b923728d27f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:36.991593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:36.998188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:37.000795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:37.002300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:37.004773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 299/606] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:37.006023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:37.011724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 308/607] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:59:36 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/c95553acbf23423993a4d25e83ecee48/roles/adad88d8a75747a381a2b923728d27f8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 13:59:37.012851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:37.225144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce03f96-f775-490a-ae78-8e2be2547699 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:37.244995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671014b8-ac79-4044-ba34-f45d43574210 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:37.254585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:37.286407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:37.287019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:37.287295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:37.287560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:37.288245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:37.288545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:37.290002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:37.296272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:37.297506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:37.297827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:37.298112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:37.303988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 309/608] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:37.309568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:37.310441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:37.317633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:37.318694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:37.319542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 300/609] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:37.325984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:37.372008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:37.470441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e4cf5-63dc-43dd-b004-563d95898615 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:37.516882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125df9-e235-43af-986c-1243f302430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:37.518880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125df9-e235-43af-986c-1243f302430e None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:37.520506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125df9-e235-43af-986c-1243f302430e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:37.526525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b125df9-e235-43af-986c-1243f302430e None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/11a418a2a35146b59c433421fe507500/roles/adad88d8a75747a381a2b923728d27f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:37.550545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc74e-cf21-4da8-8ab9-c1e4a39c3060 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:37.556009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125df9-e235-43af-986c-1243f302430e None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=11a418a2a35146b59c433421fe507500, role_id=adad88d8a75747a381a2b923728d27f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:37.557721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b125df9-e235-43af-986c-1243f302430e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:37.592639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:37.593398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:37.593849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:37.594217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:37.594980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:37.595384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:37.596337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 310/610] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:59:37 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/11a418a2a35146b59c433421fe507500/roles/adad88d8a75747a381a2b923728d27f8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:37.601758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:37.616263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 301/611] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:37.622782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125df9-e235-43af-986c-1243f302430e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:37.664729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:37.756920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b125df9-e235-43af-986c-1243f302430e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:37.813623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:37.814359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:37.814637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:37.814910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:37.826223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:37.828434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:37.843709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:37.844698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:37.845583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:37.845583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:37.845583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:37.845583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 311/612] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:37.850612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:37.949078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:37.962348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5a8d-0314-431e-93dc-887e9dfdbbe1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:37.998359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:37.999647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.000010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.000282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.000929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:38.001228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:38.002669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:38.015405 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 302/613] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:38.020816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:38.107723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:38.137984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988858b4-d6bb-4e9b-b953-5bf3cff472f1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:38.187012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:38.187642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.187916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.188187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.195993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:38.197426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:38.209890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:38.212119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:38.214458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 312/614] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:38.218319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:38.324995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825243ac-96bb-4f1d-becd-64a3da4f2806 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:38.361926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:38.362404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.362680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.362951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.363677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:38.363984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:38.365395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:38.384038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 303/615] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 13 13:59:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:38.389750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:38.503394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3092a50-f9c6-43c5-8b0c-4832f59a319c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:38.538163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961bf81-3da8-40a9-9c78-559afa20186b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:38.539286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961bf81-3da8-40a9-9c78-559afa20186b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.539560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961bf81-3da8-40a9-9c78-559afa20186b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.539829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961bf81-3da8-40a9-9c78-559afa20186b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.548682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:38.553915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961bf81-3da8-40a9-9c78-559afa20186b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:38.556002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961bf81-3da8-40a9-9c78-559afa20186b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6961bf81-3da8-40a9-9c78-559afa20186b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:38.572087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:38.573132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:38.574528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:38.574528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:38.574528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:38.574528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 313/616] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:38.579978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:38.589270 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:38.662963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b10aaf-94b4-450e-9133-6c26a9dc4784 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.663261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b10aaf-94b4-450e-9133-6c26a9dc4784 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.663452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b10aaf-94b4-450e-9133-6c26a9dc4784 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.665004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 314/617] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:59:38.669940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad904d1-c968-4614-85ab-e95a1e512e0b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.670223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad904d1-c968-4614-85ab-e95a1e512e0b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.670495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad904d1-c968-4614-85ab-e95a1e512e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.686811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e36bb-9892-48ed-9903-801165d8c3be None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:38.690425 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bad904d1-c968-4614-85ab-e95a1e512e0b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:38.691620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bad904d1-c968-4614-85ab-e95a1e512e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:38.725235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:38.726372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.726660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.728508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.729168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:38.729466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:38.731748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:38.761627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 304/618] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:38.765576 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:38.846312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 315/619] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:38.866165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:38.900118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c37bd-bc40-4586-8a7d-40c4772822d5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:38.935032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:38.935581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:38.935856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:38.936127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:38.943818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:38.945305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:38.956747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:38.958188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:38.959656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 305/620] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:38.963955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad904d1-c968-4614-85ab-e95a1e512e0b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:39.103739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad904d1-c968-4614-85ab-e95a1e512e0b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:39.124934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:39.140132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57d406-4651-426d-a81c-6d2c5b0ee300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:39.140785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d406-4651-426d-a81c-6d2c5b0ee300 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.141061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d406-4651-426d-a81c-6d2c5b0ee300 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.141327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d406-4651-426d-a81c-6d2c5b0ee300 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.142137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d406-4651-426d-a81c-6d2c5b0ee300 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:39.142436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d406-4651-426d-a81c-6d2c5b0ee300 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:39.143849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d406-4651-426d-a81c-6d2c5b0ee300 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:39.156750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 316/621] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:39.162494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:39.205715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c72bd-ab78-49ab-b0ca-73d527edcbbf None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.206055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c72bd-ab78-49ab-b0ca-73d527edcbbf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.206330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c72bd-ab78-49ab-b0ca-73d527edcbbf None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.207656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 317/622] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:59:39.212519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9096-9d06-4f22-93cd-3316b71bfbbf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.212803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9096-9d06-4f22-93cd-3316b71bfbbf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.213073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc9096-9d06-4f22-93cd-3316b71bfbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.235929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-73cc9096-9d06-4f22-93cd-3316b71bfbbf None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:39.237268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73cc9096-9d06-4f22-93cd-3316b71bfbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:39.283964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dfdb0-2902-4f2d-a79e-a36a1fdb4640 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:39.316351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:39.326416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 318/623] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:39.327627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc9096-9d06-4f22-93cd-3316b71bfbbf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:39.328409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:39.329334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.329858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.330390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/15e78b0e48a141edad05f3c064ac04f7/roles/adad88d8a75747a381a2b923728d27f8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.346854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=15e78b0e48a141edad05f3c064ac04f7, role_id=adad88d8a75747a381a2b923728d27f8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:39.348657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:39.366744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 306/624] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:59:39 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/15e78b0e48a141edad05f3c064ac04f7/roles/adad88d8a75747a381a2b923728d27f8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:39.485558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc9096-9d06-4f22-93cd-3316b71bfbbf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:39.502259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:39.543298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:39.543976 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.544255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.544525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.563664 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:39.568799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:39.569842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:39.571103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 319/625] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:39.578837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:39.694769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2b23d-c0ab-40f1-b257-480b4709c84a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:39.737670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:39.738259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.738538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.738810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.739498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:39.739802 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:39.741224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:39.760477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 307/626] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:39.767213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:39.786151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:39.840908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:39.909372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f37d98-39b9-43de-b8f4-7d046a11ae25 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:39.917228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5a38f-bcc8-4cca-9c65-ea069ec4bbc3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:39.951214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:39.952754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.953294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.953561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.957427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff20885-aaa8-4d0b-aee9-e9a97119161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:39.958153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff20885-aaa8-4d0b-aee9-e9a97119161c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.958505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff20885-aaa8-4d0b-aee9-e9a97119161c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.958850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff20885-aaa8-4d0b-aee9-e9a97119161c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.965264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:39.966844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:39.970895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff20885-aaa8-4d0b-aee9-e9a97119161c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-aff20885-aaa8-4d0b-aee9-e9a97119161c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:39.975811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:39.976965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:39.978067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:39.979164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 308/627] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:39.982249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203f3b3-64b5-458f-8c3e-2eaa8bbeef9e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.982249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203f3b3-64b5-458f-8c3e-2eaa8bbeef9e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.982249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203f3b3-64b5-458f-8c3e-2eaa8bbeef9e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.986881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 309/628] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:39.988237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc574-c883-4e17-9b7d-c7156d5fd979 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:39.988474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc574-c883-4e17-9b7d-c7156d5fd979 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:39.988753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc574-c883-4e17-9b7d-c7156d5fd979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:39.990655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:39.991669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:39.992587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:39.992587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:39.992587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:39.994589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 320/629] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:39.995615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.012011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4e7bc574-c883-4e17-9b7d-c7156d5fd979 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:40.013558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4e7bc574-c883-4e17-9b7d-c7156d5fd979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:40.028009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:40.120690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 310/630] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:40.120690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bc574-c883-4e17-9b7d-c7156d5fd979 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.125206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f7f32-56f9-421b-a3d3-d51f6f7b9f9f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.172800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:40.184527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.185136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.187839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.188134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.188813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:40.189115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:40.194093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:40.210790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 321/631] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:40.213918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.305387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bc574-c883-4e17-9b7d-c7156d5fd979 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.372398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:40.372620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.373202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.373479 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.374329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.374981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:40.375356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:40.378896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:40.400257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 311/632] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:40.403800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11003483-f86f-4efb-8bdb-187f8088c43b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.405924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.463751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe35110-4d0f-4caa-9879-b13059414cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.464503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe35110-4d0f-4caa-9879-b13059414cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.464789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe35110-4d0f-4caa-9879-b13059414cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.465064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe35110-4d0f-4caa-9879-b13059414cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.477364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe35110-4d0f-4caa-9879-b13059414cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:40.478845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe35110-4d0f-4caa-9879-b13059414cd7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8fe35110-4d0f-4caa-9879-b13059414cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:40.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:40.498101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:40.499564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:40.499564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:40.499564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:40.499564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 322/633] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:40.500489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1e3f-d00d-46a3-97b5-20f3c2c6a508 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.500784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1e3f-d00d-46a3-97b5-20f3c2c6a508 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.501061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc1e3f-d00d-46a3-97b5-20f3c2c6a508 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.502456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 323/634] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:40.506886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1e3f-d00d-46a3-97b5-20f3c2c6a508 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.537706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:40.552682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43419a1b-0746-48a8-af63-cc3e17f3d5a4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.589779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.590449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.590725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.590995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.598536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:40.599999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:40.612775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:40.613782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 312/635] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:40.614749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.618939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc1e3f-d00d-46a3-97b5-20f3c2c6a508 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.659838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185701a-f0f9-4285-b0f0-bab3e5bbbaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.660574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185701a-f0f9-4285-b0f0-bab3e5bbbaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.660848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185701a-f0f9-4285-b0f0-bab3e5bbbaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.661118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185701a-f0f9-4285-b0f0-bab3e5bbbaa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.664229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:40.673821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185701a-f0f9-4285-b0f0-bab3e5bbbaa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0185701a-f0f9-4285-b0f0-bab3e5bbbaa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:40.679298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:40.680245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:40.681101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 324/636] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:40.682413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0abbb-f741-4d6f-a2c6-6568d01b35a8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.682413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0abbb-f741-4d6f-a2c6-6568d01b35a8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.682413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0abbb-f741-4d6f-a2c6-6568d01b35a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.699390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4ac0abbb-f741-4d6f-a2c6-6568d01b35a8 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:40.700571 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4ac0abbb-f741-4d6f-a2c6-6568d01b35a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:40.731912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe412713-37f6-4631-aabf-343754fbad1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.768031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.768641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.768923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.769196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.769873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:40.770178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:40.771528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:40.776872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 325/637] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:40.778008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0abbb-f741-4d6f-a2c6-6568d01b35a8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.796821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 313/638] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:40.797912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.809914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:40.910607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0abbb-f741-4d6f-a2c6-6568d01b35a8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.923665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f04a2e-c736-410e-80ec-74780e0e28f6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:40.936846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:40.947642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.948445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.948866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.949280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.950340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:40.950799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:40.952523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:40.966569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:40.967291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:40.968197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:40.968564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:40.969341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:40.969772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:40.971703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:40.975068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 326/639] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:40.976095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:40.986673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 314/640] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:40.987587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.075498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.090376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e999e-703f-4160-89d0-7972d0e511ce None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.110912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ae5a7-0668-47a3-9b61-38868dde812f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.127759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.128388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.128665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.128933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.136250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.137685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.148898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.149499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.149795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.150065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:41.150177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:41.151040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 327/641] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:41.151993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.201623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.229573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.231083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.263763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466c63d-d7fc-43db-a3f0-e6d2e6bbea98 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.299584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.301744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.302446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.302733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.303077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.314876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.316337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:41.332443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:41.333462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:41.334429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:41.334429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:41.334429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:41.337319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 315/642] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:41.339203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 328/643] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:41.339330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.339444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.442676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.444082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366ac9c-d672-42f3-babb-b910c64b916c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.448883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce946b6-5b36-442f-921b-be6bb864f53b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.480225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.480767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.481053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.481314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.486970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.487570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.487850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.488120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.488842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:41.489147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.490608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.492633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:41.497578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:41.498563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 316/644] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:41.500511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.520175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 329/645] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:41.520175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.547918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.627690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479db1ca-ffb3-49ac-8fcb-cbf993dd0639 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.632491 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc254945-eea3-4ea1-b791-097542de393f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.667268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.667877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.668026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.668197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.668415 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.668513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.668744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.669016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.669653 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:41.669986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.671795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.680656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.682451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.683036 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 330/646] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:41.683844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:41.703975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:41.704980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:41.705902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:41.705902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:41.705902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:41.706094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.707295 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 317/647] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:41.708861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.792188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.795179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64d049-3e76-469b-a5be-d7cf4af690d4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.812391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4a7e1-9797-4f28-8caa-f71d34d6194a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.829174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.829825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.830139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.830443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.831135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:41.831468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.832886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.841070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 331/648] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:41.841908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:41.847760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f574e0-5957-4ef0-867e-c62b1d15af52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.848361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f574e0-5957-4ef0-867e-c62b1d15af52 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.848637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f574e0-5957-4ef0-867e-c62b1d15af52 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.848908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f574e0-5957-4ef0-867e-c62b1d15af52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.915027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f574e0-5957-4ef0-867e-c62b1d15af52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.916469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f574e0-5957-4ef0-867e-c62b1d15af52 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.918474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.942074 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462db372-d4f0-4545-ae06-457d10f5f730 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:41.975610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:41.976257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:41.976576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:41.976888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:41.984905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:41.986406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:41.994786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:41.997647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:41.998629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 332/649] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:41.999607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.021041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 318/650] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:42.021041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6811a-8f47-4ab8-9854-092ba122226e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.021041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6811a-8f47-4ab8-9854-092ba122226e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.021041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6811a-8f47-4ab8-9854-092ba122226e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.021041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 319/651] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:42.021041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6811a-8f47-4ab8-9854-092ba122226e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.102712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ef345-26c4-4d06-a249-0bb52c33bb32 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.121077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6811a-8f47-4ab8-9854-092ba122226e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.136120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.136737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.137047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.137350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.138045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:42.138376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.139795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.142152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.152491 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 333/652] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:42.153661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.155907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.156539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.156820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.157097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.167325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.168766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:42.183517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:42.184658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:42.185779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:42.185779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:42.185779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:42.185779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 320/653] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:42.185779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.219421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.265218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8e49f-68dd-429e-95be-d2586507bf1e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.298094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca632e-64f5-40e9-9b73-3f4750cba08f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.301554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1c89-9aba-4769-9caa-570c6436f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.302199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1c89-9aba-4769-9caa-570c6436f46d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.302513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1c89-9aba-4769-9caa-570c6436f46d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.302818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1c89-9aba-4769-9caa-570c6436f46d None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.303575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1c89-9aba-4769-9caa-570c6436f46d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:42.303913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1c89-9aba-4769-9caa-570c6436f46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.305373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1c89-9aba-4769-9caa-570c6436f46d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.322663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 334/654] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:42.323936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6358ca-d8ea-4624-b10b-b64e2c3393e1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.324222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6358ca-d8ea-4624-b10b-b64e2c3393e1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.324495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6358ca-d8ea-4624-b10b-b64e2c3393e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.335570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.336195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.336478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.336749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.342614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1d6358ca-d8ea-4624-b10b-b64e2c3393e1 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:42.344282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1d6358ca-d8ea-4624-b10b-b64e2c3393e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:42.347993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.349415 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:42.363404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:42.364514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:42.365606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:42.365606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:42.365606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:42.368872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 321/655] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:42.369694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.394459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.412193 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 335/656] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:42.413140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6358ca-d8ea-4624-b10b-b64e2c3393e1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.455637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.471437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941225d4-06ea-4bf9-8796-46df443d28bc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.507942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.508667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.509020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.509569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.518071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.520125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.531630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6358ca-d8ea-4624-b10b-b64e2c3393e1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:42.535292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:42.536151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:42.537043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:42.541379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 322/657] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:42.542068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.568088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.568838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.569215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.569597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.570362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:42.570720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.572101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.584949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 336/658] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:42.586184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.638988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.655773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7939a7-bb0e-4b83-a0c1-fdf6f8ebdb71 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.690788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.691390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.691667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.691938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.694748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c8f0e-d4f4-4428-9737-472ac687b5f6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.707285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.728251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.728845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.729122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.729391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.730051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:42.730349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.731702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.743925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 337/659] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:42.744738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.759328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.760762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.844684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474620d-8866-44af-866f-ab2622f6bd13 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:42.846955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.860830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 323/660] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:42.861665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.878841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:42.879445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:42.879722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:42.879991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:42.880678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:42.880979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:42.882456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:42.898013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 338/661] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:42.898013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:42.921262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:42.975305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd33e34-f070-40cd-bf90-79068b2a1778 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.004519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74671de0-3015-4f2a-8296-0e002c837dfb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.009487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.010141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.010416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.010686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.017702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.019145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:43.031202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:43.032356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 324/662] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:43.033317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.040781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.041419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.041700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.042006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.052722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:43.057990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:43.059223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:43.060071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 339/663] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:43.060792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.086662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.135076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1aa6b-6731-44e2-a1cf-af5a778c5df5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.142018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.161413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377e08-3647-4cfc-8ade-bc9c7a1278f5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.171466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.172084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.172421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.172699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.179208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.180640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:43.191650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:43.192533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 325/664] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:43.194033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.198377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.199008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.199283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.199547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.211124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.212564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:43.227625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:43.228747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:43.230110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:43.230110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:43.230110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:43.230110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 340/665] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:43.230110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.297221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706589c-16e3-4881-80c4-1c04b4581c51 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.305190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.329236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f95b1-b7d2-48fb-8f6b-6ff158ecff23 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.331523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.332126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.332401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.332669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.333382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=f7214e6addc646a899efb82283ededc1) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:43.333683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.335112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.340776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 326/666] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/role_assignments?user.id=f7214e6addc646a899efb82283ededc1&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:43.341898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.375740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.383767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.384394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.384672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.384943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.386281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:43.386595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.388514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.419649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 341/667] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:43.419649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.450887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2c22-34c2-4211-a196-5506f52652af None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.491218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.491840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.492123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.492391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.493065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:43.493366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.494784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.504687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 327/668] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:43.505384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.555688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c659f-23e9-40fa-8ed5-dd0cbc01ed48 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.598415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.599037 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.599319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.599595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.600380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:43.600692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.602860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.604743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.616493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 342/669] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:43.617180 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.618833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.629787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1e3bd-b6c1-4519-ae6a-0241d9ef012f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.666664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.667283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.667596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.667867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.668515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:43.668819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.670277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.684117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 328/670] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:43.685285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.726051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacdca2-5191-4d96-a5c5-3cef261ab7e0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.761135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.761763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.762046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.762318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.790994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5853d-f3eb-4a85-8aa0-72fede9da259 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:43.824474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.826613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:43.827885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:43.828153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:43.828419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:43.834308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.835989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.840344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:43.841764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:43.856590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:43.857592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:43.859550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:43.859550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:43.859550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:43.859550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 329/671] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:43.859550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.863068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:43.948910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 343/672] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:43.949831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:43.968003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d48eec-c8e6-4c42-a7b5-b241cacee8cf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.002560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.003160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.003443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.003719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.004435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=6a4ea803f6e44648bb217abd00e11c3c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:44.004750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.006205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.011847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 330/673] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/role_assignments?user.id=6a4ea803f6e44648bb217abd00e11c3c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:44.012610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.069621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:44.078937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bb23b-940b-4b2b-a435-634a60ff2113 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.098711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:44.134814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.135569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.135917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.136519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.144217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.145718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.147710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d92405-7ace-48b0-aeb3-69a200062caa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:44.158856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:44.159819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:44.160718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 344/674] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:44.161744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.190055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.190831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.191121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.191551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.192270 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:44.192580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.194191 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.208977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 331/675] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:44.209813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.276768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512826f8-bff0-45f6-8d87-ef4d37a56003 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.317247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.318353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.318854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.319330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.325112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31405f3c-8e60-43cc-81fb-4a353836d2e3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.334132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.335987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:44.355072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:44.356067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:44.356952 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:44.356952 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:44.356952 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:44.356952 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 345/676] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:44.357624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.365125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:44.367303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:44.370842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.371458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.371744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.372015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.372789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:44.373098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.375605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.388462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 332/677] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:44.389114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.461448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821ae6c-5e84-4597-9e36-299e3942c870 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.496990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.497615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.497924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.498194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.498438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb250998-42a4-471a-b19f-6823531c6c69 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.506470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.507907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:44.520742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:44.521773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:44.522782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 346/678] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:44.523374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.543781 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.544745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.545340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.546361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.547284 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:44.547795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.549439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.563036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 333/679] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:44.564205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.595502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:44.620805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:44.634627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ef47d-5f9d-4cbf-bfa7-220427124cec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.671821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.672433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.672710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.672984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.673788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=c95553acbf23423993a4d25e83ecee48) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:44.674105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.675647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.679387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c4637-d230-45ff-88e5-97ba44694081 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.682691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 347/680] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/role_assignments?user.id=c95553acbf23423993a4d25e83ecee48&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:44.683403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.719507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.720513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.720995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.721489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.722475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:44.723034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.724765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.760223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 334/681] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:44.765626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.795235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fba63-670f-408a-a963-8fc8394f548a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.829674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:44.835002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:44.835651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:44.835935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:44.836192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:44.852776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:44.854280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:44.882977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:44.883948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:44.885217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 348/682] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:44.890115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:44.964791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5aad3b-091f-4040-8940-2fab84c34b55 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:44.976530 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:45.012892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:45.016230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.016230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.016230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.034714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08c32b-97b8-42ee-8f59-90517153645f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:45.072878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:45.073484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.073800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.074079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.074839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:45.075143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:45.076601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:45.090219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 349/683] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:45.097877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:45.119154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:45.120899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:45.171198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:45.226158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c675954-6d72-49a9-aeb2-a3a4c3e8bef9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:45.281894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:45.282420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.282696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.282965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.283691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=11a418a2a35146b59c433421fe507500) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:45.283995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:45.285422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:45.296778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 350/684] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:45 2022] GET /identity/v3/role_assignments?user.id=11a418a2a35146b59c433421fe507500&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:45.316746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 335/685] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:45.321991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:45.465033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4215f2-8fe3-4a37-ab39-6f4da2af82c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:45.502303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:45.519127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a651c-514a-4a53-b638-b188fd6dcced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:45.519792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a651c-514a-4a53-b638-b188fd6dcced None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.520077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a651c-514a-4a53-b638-b188fd6dcced None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.520357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a651c-514a-4a53-b638-b188fd6dcced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.542464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a651c-514a-4a53-b638-b188fd6dcced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:45.542464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a651c-514a-4a53-b638-b188fd6dcced None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-722a651c-514a-4a53-b638-b188fd6dcced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:45.552511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:45.553524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:45.554485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:45.554485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:45.554485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:45.554485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 351/686] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:45.563301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:45.574645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:45.707303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b244a13-afdb-4106-805d-5fa6e74c849e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:45.743115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:45.743772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.744052 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.744379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.745026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:45.745329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:45.746711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:45.753598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944d8f7-840c-4011-b52f-63d99b3cec39 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.754232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944d8f7-840c-4011-b52f-63d99b3cec39 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.754232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944d8f7-840c-4011-b52f-63d99b3cec39 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.755557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 352/687] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:59:45.761673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 336/688] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:45.761973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f95cd-6b2e-49f8-bdbe-2912ea218a21 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.762137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f95cd-6b2e-49f8-bdbe-2912ea218a21 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.762337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f95cd-6b2e-49f8-bdbe-2912ea218a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.767448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:45.783460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d72f95cd-6b2e-49f8-bdbe-2912ea218a21 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:45.784719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d72f95cd-6b2e-49f8-bdbe-2912ea218a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:45.843754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:45.888313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 353/689] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:45.892522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd56ae-4675-4e57-8ccb-bccae60c32d0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:45.940804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:45.941836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:45.942316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:45.942785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:45.952997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:45.954719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:45.974009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:45.974958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:45.975845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:45.977904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 337/690] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:45.984242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f95cd-6b2e-49f8-bdbe-2912ea218a21 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:46.093659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:46.100106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f95cd-6b2e-49f8-bdbe-2912ea218a21 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:46.143989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:46.144598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:46.144873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:46.145148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:46.145877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:46.146178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:46.147595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:46.160651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 354/691] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:46.166491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:46.189906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:46.307898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e9346-351b-452d-b01e-ee96ae051a80 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:46.342179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f5c7c-ef47-4094-9a5d-0113ee164745 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:46.369433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:46.371215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:46.371822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:46.373035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:46.394046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:46.395065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:46.397388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:46.417199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:46.418298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:46.419203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:46.419203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:46.419203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:46.419381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:46.419939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:46.420229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:46.420543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:46.421369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=15e78b0e48a141edad05f3c064ac04f7) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:46.421680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:46.422570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 355/692] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:46.427470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:46.428772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:46.430060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 338/693] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:46 2022] GET /identity/v3/role_assignments?user.id=15e78b0e48a141edad05f3c064ac04f7&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:46.448341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:46.590002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88b10a-fb54-4208-b5a7-0188c3d1a294 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:46.646199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26a85-3946-483e-a229-b1e9fe838238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:46.646719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26a85-3946-483e-a229-b1e9fe838238 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:46.646995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26a85-3946-483e-a229-b1e9fe838238 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:46.647263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26a85-3946-483e-a229-b1e9fe838238 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:46.647902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26a85-3946-483e-a229-b1e9fe838238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:46.648201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26a85-3946-483e-a229-b1e9fe838238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:46.649557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26a85-3946-483e-a229-b1e9fe838238 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:46.662730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 356/694] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:46.668197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:46.730186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae36e-2581-411e-bc41-235aed5d4f84 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:46.730811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae36e-2581-411e-bc41-235aed5d4f84 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:46.731393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aae36e-2581-411e-bc41-235aed5d4f84 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:46.732913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 357/695] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:59:46.738774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b4526-5df0-4b55-b427-6309a5a10ae1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:46.739389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b4526-5df0-4b55-b427-6309a5a10ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:46.739730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b4526-5df0-4b55-b427-6309a5a10ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:46.760575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c33b4526-5df0-4b55-b427-6309a5a10ae1 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:46.762584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c33b4526-5df0-4b55-b427-6309a5a10ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:46.786788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c40660-2b2c-431f-8115-87d2868e2cb3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:46.833745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:46.834631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:46.834984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:46.836119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:46.836289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:46.838119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:46.842530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:46.845356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 358/696] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:46.866139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 339/697] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 13:59:46 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:47.137130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b4526-5df0-4b55-b427-6309a5a10ae1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:47.252421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b4526-5df0-4b55-b427-6309a5a10ae1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:47.290045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:47.291106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:47.291635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:47.292132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:47.303951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:47.309232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:47.310298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:47.311919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:47.314517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 359/698] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:47.320450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:47.372352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:47.435990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90107e4c-781c-4ffd-90b3-ead560b296c2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:47.477207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0aed2c-da98-4296-a8e4-83d047bea689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:47.477840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aed2c-da98-4296-a8e4-83d047bea689 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:47.478125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aed2c-da98-4296-a8e4-83d047bea689 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:47.478400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0aed2c-da98-4296-a8e4-83d047bea689 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:47.479073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0aed2c-da98-4296-a8e4-83d047bea689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:47.479376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0aed2c-da98-4296-a8e4-83d047bea689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:47.481022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0aed2c-da98-4296-a8e4-83d047bea689 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:47.490700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 340/699] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:47.496518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:47.628804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:47.671145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27db40-446c-4680-8a88-ee65d83af7d9 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:47.671663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27db40-446c-4680-8a88-ee65d83af7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:47.671968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27db40-446c-4680-8a88-ee65d83af7d9 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:47.674629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 341/700] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:47.678820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cb6a8-7e40-4273-bd13-c14323a86fcc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:47.679107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cb6a8-7e40-4273-bd13-c14323a86fcc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:47.679384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cb6a8-7e40-4273-bd13-c14323a86fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:47.698415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b232f-2247-43a9-9dd7-c8cbfd4fb1e6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:47.700541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9d7cb6a8-7e40-4273-bd13-c14323a86fcc None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:47.701847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9d7cb6a8-7e40-4273-bd13-c14323a86fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:47.740022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:47.740660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:47.740937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:47.741205 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:47.762087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:47.763539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:47.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:47.794825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:47.796357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:47.796357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:47.796357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:47.798208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 360/701] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:47.803570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:47.841295 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 342/702] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:47.941299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff9ee3-abf5-4265-9011-3dc5ce4af6f9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:47.953512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:47.978766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:47.979789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:47.980315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:47.980787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:47.981755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:47.982278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:47.983899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:47.996503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 361/703] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:48.002960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cb6a8-7e40-4273-bd13-c14323a86fcc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:48.053319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:48.127836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cb6a8-7e40-4273-bd13-c14323a86fcc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:48.176760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901fc15-3aec-4610-9ae9-2c2f88386637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:48.177428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901fc15-3aec-4610-9ae9-2c2f88386637 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.177706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901fc15-3aec-4610-9ae9-2c2f88386637 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.178002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901fc15-3aec-4610-9ae9-2c2f88386637 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.188564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901fc15-3aec-4610-9ae9-2c2f88386637 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:48.190063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901fc15-3aec-4610-9ae9-2c2f88386637 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0901fc15-3aec-4610-9ae9-2c2f88386637 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:48.208999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:48.210086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:48.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 343/704] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:48.216024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:48.312544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d104ec-f92b-483e-b781-7180751f3af8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.313179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d104ec-f92b-483e-b781-7180751f3af8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.313650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d104ec-f92b-483e-b781-7180751f3af8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.316545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 344/705] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:48.320800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2281-efc1-4043-a98e-392737d0ca52 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.321786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2281-efc1-4043-a98e-392737d0ca52 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.322262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b2281-efc1-4043-a98e-392737d0ca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.340592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d96b2281-efc1-4043-a98e-392737d0ca52 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:48.342131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d96b2281-efc1-4043-a98e-392737d0ca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:48.357480 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86a613-3d31-4c0f-a625-06b6973706da None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:48.419909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:48.419909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.419909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.419909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.420329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:48.420329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:48.420329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:48.421014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 345/706] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 13:59:48.422242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b2281-efc1-4043-a98e-392737d0ca52 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:48.440142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 362/707] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:48.445254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:48.480102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:48.491665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:48.578257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164abc47-ae41-4867-ac5d-e19fa3035215 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:48.617913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:48.619019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.619370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.619715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.672589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b2281-efc1-4043-a98e-392737d0ca52 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:48.693516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:48.695102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:48.729416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:48.735555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:48.736775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.737055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.737316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.737936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:48.739292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:48.751934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:48.752982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:48.753840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 346/708] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:48.754689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:48.817374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 363/709] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 13:59:48.819033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:48.873345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:48.883745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d94211-f725-4d43-8085-7d6698f4c921 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:48.920999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:48.921755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.922037 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.922312 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.922895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:48.924252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:48.931433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3393bf-02ca-4fd2-bc81-6a370c1b84c0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:48.933687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:48.934702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 13:59:48.935554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:48.939680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 347/710] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:48.940543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:48.965023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:48.975246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:48.976233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:48.976842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:48.977316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:48.978212 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:48.978729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:48.980328 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:48.986575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 364/711] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:48.987806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:49.047121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1efda3-53df-44a5-86c5-f253a28f74d5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:49.086574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:49.087686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.088214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.088730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.101799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:49.103823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:49.110399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890be71-ca5b-4a9c-87cc-3ffae3d54bc5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:49.122149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:49.123202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:49.124323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:49.124323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:49.124323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:49.128293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 348/712] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:49.129376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:49.141807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:49.160622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:49.161716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.162502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.163205 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.164007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] RBAC: Adding query filter params (name=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:49.164637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:49.166812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:49.176808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 365/713] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 13:59:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:49.177971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:49.242503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:49.260238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7cd1a-48b1-4d5b-a4fa-a211b9d2b0a2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:49.290197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0886f-6925-4e33-be46-95975e26a258 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:49.310747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:49.312435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.312991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.313469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.315095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:49.315667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:49.317897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:49.328204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 349/714] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 13 13:59:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:49.330030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:49.337761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:49.337761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.337761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.337761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.338196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:49.338196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:49.338196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:49.353784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 366/715] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:49.359583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:49.430260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:49.468943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21161535-68aa-4adb-b0ce-cec6e9ccbf22 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:49.511178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842cc62-7814-43de-9b1e-24aa6b75f8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:49.512724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842cc62-7814-43de-9b1e-24aa6b75f8bf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.513495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842cc62-7814-43de-9b1e-24aa6b75f8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.514263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842cc62-7814-43de-9b1e-24aa6b75f8bf None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.515183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842cc62-7814-43de-9b1e-24aa6b75f8bf None admin] RBAC: Adding query filter params (type=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:49.515703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842cc62-7814-43de-9b1e-24aa6b75f8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:49.515908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc5ca0-fbe6-4c9a-bd0f-682408b5427c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:49.517524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842cc62-7814-43de-9b1e-24aa6b75f8bf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:49.533700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 350/716] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 13:59:49 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:49.533700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7bff0-ead9-4ef2-8afa-ae448c8dfbe6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.533700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7bff0-ead9-4ef2-8afa-ae448c8dfbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.533700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7bff0-ead9-4ef2-8afa-ae448c8dfbe6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.534309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 351/717] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:49.540771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7a8fa-383b-4081-8bf1-3885e1f0f933 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.541080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7a8fa-383b-4081-8bf1-3885e1f0f933 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.541883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7a8fa-383b-4081-8bf1-3885e1f0f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.545262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:49.573262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f2e7a8fa-383b-4081-8bf1-3885e1f0f933 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:49.574686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f2e7a8fa-383b-4081-8bf1-3885e1f0f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:49.575926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:49.577312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.578122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.578473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.586337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:49.587940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:49.602316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:49.603343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:49.604267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 367/718] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:49.610556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:49.667952 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 352/719] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:49.763922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139f297-ea97-4313-835a-e56e9e6e2319 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:49.793302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b429f3-0679-4955-ab6b-d7dd8bb27702 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.800147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b429f3-0679-4955-ab6b-d7dd8bb27702 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.800766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b429f3-0679-4955-ab6b-d7dd8bb27702 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.802739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 353/720] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:49.808754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd024e2-c130-4927-89d6-2cb4fe63570c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.809338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd024e2-c130-4927-89d6-2cb4fe63570c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.811333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd024e2-c130-4927-89d6-2cb4fe63570c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.854699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5cd024e2-c130-4927-89d6-2cb4fe63570c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:49.855502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:49.856150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:49.856431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:49.856705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:49.857400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:49.857694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5cd024e2-c130-4927-89d6-2cb4fe63570c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:49.857824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:49.859804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:49.900554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 368/721] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:49.911739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:49.950127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:49.962152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 354/722] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:49.989340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:50.063022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd024e2-c130-4927-89d6-2cb4fe63570c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:50.119846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111ec92-0dd0-47ce-9282-00c4281fa0f2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:50.223846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:50.224831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:50.225313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:50.226713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:50.227652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=6a4ea803f6e44648bb217abd00e11c3c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:50.228194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:50.231674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:50.246407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 369/723] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:49 2022] GET /identity/v3/role_assignments?user.id=6a4ea803f6e44648bb217abd00e11c3c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:50.313267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd024e2-c130-4927-89d6-2cb4fe63570c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:50.328541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:50.396237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:50.396943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:50.397182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:50.397459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:50.425284 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:50.434333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:50.435349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:50.436369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:50.442566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 355/724] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:50.445823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:50.544288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:50.552564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a8f38-f238-491e-984f-cc857c7dd67e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:50.555989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:50.611802 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf2b8-493d-40b4-93e2-95b8e2e2a742 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:50.619252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:50.620216 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:50.620762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:50.621283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:50.646724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:50.648969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:50.649934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:50.650448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:50.650943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:50.651811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:50.652321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:50.654265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:50.656694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:50.657624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:50.658796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:50.664840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 370/725] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:50.666093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:50.674342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 356/726] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:50.685753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:50.852914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6a82e-75c1-441f-9978-a2fc7b52ab94 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:50.869141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261342d9-0165-42f3-abb9-19f11705983c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:50.904278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:50.927387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:50.928063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:50.928269 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:50.928541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:50.929207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:50.929506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:50.930908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:50.938015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:50.938639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:50.938914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:50.939181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:50.952896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 371/727] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:50.959710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:50.961168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:50.962715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:50.976533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:50.977532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:50.979097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:50.979097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:50.979097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:50.979097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 357/728] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:50.982945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:51.012044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:51.132411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bac2d8-39a1-4775-9793-6f3ff8ff9027 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:51.178334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edbb8f-f638-4c54-a3e2-8e3eb53d86aa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:51.201645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:51.202827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:51.203087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:51.203347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:51.225022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:51.227597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:51.239472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:51.240080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:51.240355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:51.240620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:51.241255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:51.241553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:51.244740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:51.252454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:51.254694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:51.256853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:51.256853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:51.256853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:51.256853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 372/729] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:51.262957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:51.274472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 358/730] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:51.275493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:51.334093 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:51.392065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:51.447575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be9c68-8544-460e-9092-ecc648ff30c1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:51.474739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf842c77-7868-4f7e-bb59-0a5307ca97be None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:51.518864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:51.519532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:51.519808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:51.520690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:51.521307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:51.521575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:51.521864 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:51.522394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:51.523051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:51.523348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:51.524717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:51.539516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:51.540962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:51.551694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 373/731] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:51.557274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:51.571344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:51.573592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:51.575925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 359/732] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:51.581749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:51.725660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87cbd8-9ce1-4600-a973-3c863d529ea2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:51.765406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:51.766077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:51.766330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:51.766600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:51.768688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:51.775895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:51.778219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:51.781028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:51.793259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:51.794389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 13:59:51.795347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:51.798461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 374/733] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 13:59:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:51.806710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:51.821567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5b560-0d12-47fa-9aa7-6d3a44567129 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:51.903355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:51.904054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:51.904363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:51.904647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:51.905830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:51.906171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:51.907606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:51.945439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 360/734] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 13 13:59:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:51.951150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:51.955900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9acd6-d6bd-475b-8622-d34a097c2620 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:51.994570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:51.995090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:51.995531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:51.995806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:51.996831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:51.997143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:51.998689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:52.033664 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 375/735] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 13 13:59:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:52.039291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:52.106623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:52.117573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac63e5d-af2f-48a2-91fc-5e504123a750 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:52.163428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728248ee-565f-46c7-9bd0-d1170783eb93 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:52.164963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a7f15-7c6e-4703-8c63-dff5b89b4ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:52.166583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7f15-7c6e-4703-8c63-dff5b89b4ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.167025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7f15-7c6e-4703-8c63-dff5b89b4ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.167458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7f15-7c6e-4703-8c63-dff5b89b4ab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.190634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:52.200762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a9cf1-11bc-4f33-bd0d-53632c28397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:52.201379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a9cf1-11bc-4f33-bd0d-53632c28397e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.201710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a9cf1-11bc-4f33-bd0d-53632c28397e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.202159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a9cf1-11bc-4f33-bd0d-53632c28397e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.252403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a7f15-7c6e-4703-8c63-dff5b89b4ab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:52.253807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a7f15-7c6e-4703-8c63-dff5b89b4ab4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:52.276233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a9cf1-11bc-4f33-bd0d-53632c28397e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:52.277686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a9cf1-11bc-4f33-bd0d-53632c28397e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:52.335300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:52.373917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 361/736] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 13:59:52.375597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee540-8af2-44f6-bf3a-450741957fcd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.375884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee540-8af2-44f6-bf3a-450741957fcd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.376159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adee540-8af2-44f6-bf3a-450741957fcd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.381478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 362/737] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:52.381478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d3e43-6168-40c3-ad1b-a214d988a3c6 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.381694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d3e43-6168-40c3-ad1b-a214d988a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.381912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d3e43-6168-40c3-ad1b-a214d988a3c6 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.383120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 363/738] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:52.383828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d3e43-6168-40c3-ad1b-a214d988a3c6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:52.397656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 376/739] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 13:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 13:59:52.398940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244a4de-5b30-4055-9d61-0e954833fdf7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.399225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244a4de-5b30-4055-9d61-0e954833fdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.399499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244a4de-5b30-4055-9d61-0e954833fdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.419353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2244a4de-5b30-4055-9d61-0e954833fdf7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:52.421404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2244a4de-5b30-4055-9d61-0e954833fdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:52.446888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:52.491370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 377/740] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:52.493002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373ca3-acf2-45b9-8e13-ded739dc49a9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.493510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373ca3-acf2-45b9-8e13-ded739dc49a9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.494009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373ca3-acf2-45b9-8e13-ded739dc49a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.512005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d3e43-6168-40c3-ad1b-a214d988a3c6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:52.514349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-69373ca3-acf2-45b9-8e13-ded739dc49a9 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:52.516503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-69373ca3-acf2-45b9-8e13-ded739dc49a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:52.556543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:52.557603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.558152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.558652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.570827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:52.572653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:52.588502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:52.589520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:52.590483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:52.590483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:52.590483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:52.597067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 364/741] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:52.598055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 378/742] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:52.598544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:52.600495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373ca3-acf2-45b9-8e13-ded739dc49a9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:52.631630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:52.774121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:52.778241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80138a35-9950-4270-882c-09eed6ec52ab None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:52.792527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373ca3-acf2-45b9-8e13-ded739dc49a9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:52.814526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:52.815176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.815456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.815724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.826355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:52.828507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:52.828788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47106f10-912a-4a84-979d-8225d4b910f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:52.829576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47106f10-912a-4a84-979d-8225d4b910f7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.829883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47106f10-912a-4a84-979d-8225d4b910f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.830300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47106f10-912a-4a84-979d-8225d4b910f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.830796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47106f10-912a-4a84-979d-8225d4b910f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:52.831106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47106f10-912a-4a84-979d-8225d4b910f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:52.832509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47106f10-912a-4a84-979d-8225d4b910f7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:52.843812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:52.844769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:52.845674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:52.845674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:52.845674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:52.846381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 379/743] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:52.846381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa45ef-88ee-4790-9a06-3fea1ea1198e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.846381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa45ef-88ee-4790-9a06-3fea1ea1198e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.846589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa45ef-88ee-4790-9a06-3fea1ea1198e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.847212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 365/744] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:52.847966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 380/745] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:52.848421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:52.853781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa45ef-88ee-4790-9a06-3fea1ea1198e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:52.926475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:52.949897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba5505-702a-4f9a-8c8f-42a7de2890d3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:52.958521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa45ef-88ee-4790-9a06-3fea1ea1198e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:52.982986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffbc1b-606e-46bb-b727-dbc162d8213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:52.983608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffbc1b-606e-46bb-b727-dbc162d8213f None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.983890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffbc1b-606e-46bb-b727-dbc162d8213f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.984162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffbc1b-606e-46bb-b727-dbc162d8213f None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:52.984821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffbc1b-606e-46bb-b727-dbc162d8213f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:52.986231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffbc1b-606e-46bb-b727-dbc162d8213f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:52.994018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:52.994761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:52.995141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:52.995439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.002790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 366/746] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 13:59:52 2022] POST /identity/v3/services => generated 251 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:53.003773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.004067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3e6cd-dfe1-46d0-9e99-d1fed959e589 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.004355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3e6cd-dfe1-46d0-9e99-d1fed959e589 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.004627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3e6cd-dfe1-46d0-9e99-d1fed959e589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.005220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:53.018012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:53.019049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:53.019984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 381/747] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:53.020480 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.028675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e3e3e6cd-dfe1-46d0-9e99-d1fed959e589 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:53.029929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e3e3e6cd-dfe1-46d0-9e99-d1fed959e589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:53.038460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:53.126871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 367/748] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 13:59:53.127905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3e6cd-dfe1-46d0-9e99-d1fed959e589 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.150657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10be5a9-bfec-486b-b383-d3286e129699 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.183497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:53.192061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.192396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.192686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.192966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.194072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:53.194072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.195757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.215148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 382/749] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:53.216650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.265087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3e6cd-dfe1-46d0-9e99-d1fed959e589 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.309127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.310064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.310629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.310910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.311768 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.313263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.325646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:53.334643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 368/750] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 13:59:53 2022] POST /identity/v3/services => generated 240 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 13:59:53.335379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4e6bc-aebe-47cd-8c47-c400b3c65411 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.335565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.396980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.397630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.397940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.398213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.398922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:53.399226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.400684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.423631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 383/751] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:53.424800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.487707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1afc53-bd7d-47fb-9b07-c09f9f4055e8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.494796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:53.532365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.533688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.534216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.534685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.542796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.544581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:53.561198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:53.562421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:53.564254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:53.564696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a321-8ba6-4911-b5fa-a0247ea11451 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.564773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 369/752] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:53.565568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.612440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.613077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.613354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.613630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.625991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:53.631582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:53.632533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:53.633403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 384/753] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:53.634211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.655459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:53.714818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08a83b-1085-4204-971a-c0a03ec00773 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.752971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.753523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242346b4-7bd7-4bde-b235-11f111c8a425 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.753635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.753900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.754174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.754899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=15e78b0e48a141edad05f3c064ac04f7) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:53.755207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.756642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.762905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 370/754] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/role_assignments?user.id=15e78b0e48a141edad05f3c064ac04f7&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:53.763780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.766332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:53.803029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.803898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.804178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.804451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.806488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:53.807277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.810030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.826913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 385/755] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:53.833225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.885140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c794a-e4f4-471a-aa0c-9316204af0b3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:53.912247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:53.937765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:53.938465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:53.938751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:53.939020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:53.939711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:53.940013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:53.946765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:53.960314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 371/756] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:53.965070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:53.966759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b121f35-d8bb-458a-8dc9-823995982794 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:54.005376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:54.006256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:54.006681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:54.007099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:54.007957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=929edc18c5874105a349dcc2b4cad14c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:54.008372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:54.009986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:54.017258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 386/757] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/role_assignments?user.id=929edc18c5874105a349dcc2b4cad14c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:54.184846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c6d9-d5a8-444a-9db0-b662d2f1a6c8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:54.221766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:54.222419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:54.222701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:54.222976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:54.235300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:54.236794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:54.252945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:54.253977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:54.254889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:54.254889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:54.254889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:54.257924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 372/758] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:54.260364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:54.344052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:54.444535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fbe7e-65f9-41a6-b1aa-7e5c519e470f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:54.485933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:54.486537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:54.486813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:54.487089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:54.487722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:54.488023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:54.489448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:54.502344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 387/759] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:54.507710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:54.589046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:54.700582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc628-9f2b-4ec9-9762-737bef9b0613 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:54.740984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c437a-2717-427f-a491-091b043b27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:54.741621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c437a-2717-427f-a491-091b043b27ef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:54.741934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c437a-2717-427f-a491-091b043b27ef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:54.742208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c437a-2717-427f-a491-091b043b27ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:54.751064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c437a-2717-427f-a491-091b043b27ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:54.752519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c437a-2717-427f-a491-091b043b27ef None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-395c437a-2717-427f-a491-091b043b27ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:54.764647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:54.765884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 13:59:54.766835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:54.769836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 373/760] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 13:59:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:54.772118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:54.898730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:54.913363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1880c9a-cd99-4a7b-b01e-6ac384633d50 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:54.960988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:54.961530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:54.961844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:54.962118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:54.962871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:54.963179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:54.964638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:54.988683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 388/761] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 13:59:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:54.994064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c437a-2717-427f-a491-091b043b27ef None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:55.132126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:55.137597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c437a-2717-427f-a491-091b043b27ef None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:55.185936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:55.187350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:55.187651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:55.187929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:55.203433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:55.204899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:55.223641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:55.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:55.225488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:55.225488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:55.225488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:55.225488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 374/762] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:55.230454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:55.371408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894fa40-b29c-4bbf-86cb-50a175ee5c44 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:55.407792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:55.408397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:55.408673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:55.408945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:55.409581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:55.409910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:55.411264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:55.423329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 389/763] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:55.428612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:55.587658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d2a88-0385-42b1-88b0-33085d66eae2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:55.633356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2047b-ed72-4e49-a051-5b7f0210a013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:55.634036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2047b-ed72-4e49-a051-5b7f0210a013 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:55.634293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2047b-ed72-4e49-a051-5b7f0210a013 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:55.634630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2047b-ed72-4e49-a051-5b7f0210a013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:55.642268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2047b-ed72-4e49-a051-5b7f0210a013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:55.643724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2047b-ed72-4e49-a051-5b7f0210a013 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2aa2047b-ed72-4e49-a051-5b7f0210a013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:55.656041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:55.656976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:55.657921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:55.661823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 375/764] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:55.663154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:55.683752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:55.798674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ced1a-4d8c-4a3c-b7b8-c590c2906b6b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:55.812375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ded98-ef26-4734-8e49-ca84bef4c041 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:55.812946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ded98-ef26-4734-8e49-ca84bef4c041 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:55.813460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ded98-ef26-4734-8e49-ca84bef4c041 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:55.815284 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 376/765] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 13:59:55.820566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7f70-a78f-4f15-8b31-6bdf835b84c2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:55.821077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7f70-a78f-4f15-8b31-6bdf835b84c2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:55.821550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e7f70-a78f-4f15-8b31-6bdf835b84c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:55.835830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:55.836445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:55.837442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:55.837816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:55.838504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:55.838811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:55.840502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:55.843821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-525e7f70-a78f-4f15-8b31-6bdf835b84c2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:55.845226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-525e7f70-a78f-4f15-8b31-6bdf835b84c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:55.854519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 390/766] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:55.855731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:55.862761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:55.924071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 377/767] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 13:59:55.947871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:55.982399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f154e-cad7-4160-a05a-3dfbf891fc7e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:56.018727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77531cc1-ff35-4977-9b51-d0b509111634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:56.019374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77531cc1-ff35-4977-9b51-d0b509111634 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.019643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77531cc1-ff35-4977-9b51-d0b509111634 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.019914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77531cc1-ff35-4977-9b51-d0b509111634 None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/6a4ea803f6e44648bb217abd00e11c3c/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.021827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77531cc1-ff35-4977-9b51-d0b509111634 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=6a4ea803f6e44648bb217abd00e11c3c, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:56.023244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77531cc1-ff35-4977-9b51-d0b509111634 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:56.038125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 391/768] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 13:59:55 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/6a4ea803f6e44648bb217abd00e11c3c/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 13:59:56.079770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e628f-680a-43cb-8635-a6c7e4d605ba None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.079948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e628f-680a-43cb-8635-a6c7e4d605ba None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.080227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e628f-680a-43cb-8635-a6c7e4d605ba None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.081767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 378/769] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:56.087616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dbe6e-f307-4772-ac35-520cfb4125b4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.088216 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dbe6e-f307-4772-ac35-520cfb4125b4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.088718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dbe6e-f307-4772-ac35-520cfb4125b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.113371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e60dbe6e-f307-4772-ac35-520cfb4125b4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:56.114619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e60dbe6e-f307-4772-ac35-520cfb4125b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:56.208552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 392/770] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 13:59:56.252866 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e628f-680a-43cb-8635-a6c7e4d605ba None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:56.362912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e628f-680a-43cb-8635-a6c7e4d605ba None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:56.399279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:56.399905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.400183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.400452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.401034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:56.402414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:56.412546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:56.413559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:56.414489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 379/771] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Sep 13 13:59:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:56.419853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dbe6e-f307-4772-ac35-520cfb4125b4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:56.499349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:56.537090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:56.540450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:56.570704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dbe6e-f307-4772-ac35-520cfb4125b4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:56.648872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:56.649394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.649670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.649959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.650587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:56.650884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:56.652242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:56.663806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 393/772] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 13 13:59:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:56.669619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:56.742633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e6cf-7eae-4784-81c5-b4f06b9b9bf3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:56.784302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ace88-1b47-4608-ba79-fafaf6a98917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:56.784946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ace88-1b47-4608-ba79-fafaf6a98917 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.785229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ace88-1b47-4608-ba79-fafaf6a98917 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.785503 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ace88-1b47-4608-ba79-fafaf6a98917 None admin] PATH_INFO: `/v3/services/network` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.786122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ace88-1b47-4608-ba79-fafaf6a98917 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:56.787488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ace88-1b47-4608-ba79-fafaf6a98917 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-548ace88-1b47-4608-ba79-fafaf6a98917 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:56.797264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:56.798357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:56.799201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 380/773] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:56 2022] GET /identity/v3/services/network => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:56.800046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e5c0-1856-43c1-aa51-f73180e5b01d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.800632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e5c0-1856-43c1-aa51-f73180e5b01d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.800632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e5c0-1856-43c1-aa51-f73180e5b01d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.805752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 381/774] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:56.806848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8e5c0-1856-43c1-aa51-f73180e5b01d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:56.814479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7692740-a897-40cc-8b79-4c363caf68d7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:56.839953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:56.851995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c4d6b-a497-4ce6-9efd-3eeb5d8df77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:56.852602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c4d6b-a497-4ce6-9efd-3eeb5d8df77a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.852882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c4d6b-a497-4ce6-9efd-3eeb5d8df77a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.853155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c4d6b-a497-4ce6-9efd-3eeb5d8df77a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.853823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c4d6b-a497-4ce6-9efd-3eeb5d8df77a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:56.854133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c4d6b-a497-4ce6-9efd-3eeb5d8df77a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:56.855517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c4d6b-a497-4ce6-9efd-3eeb5d8df77a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:56.863121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 394/775] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 13 13:59:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:56.863121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82970e-0454-4244-924e-350a195a1805 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.863121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82970e-0454-4244-924e-350a195a1805 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.863121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82970e-0454-4244-924e-350a195a1805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.883174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2d82970e-0454-4244-924e-350a195a1805 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:56.884347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d82970e-0454-4244-924e-350a195a1805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:56.885322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:56.924839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8e5c0-1856-43c1-aa51-f73180e5b01d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:56.965753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 395/776] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 13:59:56.965753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085653-9e2a-4ed2-9e33-eca033abbb4e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.965753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085653-9e2a-4ed2-9e33-eca033abbb4e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.965753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085653-9e2a-4ed2-9e33-eca033abbb4e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.965753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 396/777] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:56.965753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf085653-9e2a-4ed2-9e33-eca033abbb4e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:56.972625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5b82c-f357-4b0d-9042-e59fa3db7283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:56.973316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5b82c-f357-4b0d-9042-e59fa3db7283 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.973674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5b82c-f357-4b0d-9042-e59fa3db7283 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.974030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5b82c-f357-4b0d-9042-e59fa3db7283 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:56.974731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5b82c-f357-4b0d-9042-e59fa3db7283 None admin] RBAC: Adding query filter params (name=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:56.975103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5b82c-f357-4b0d-9042-e59fa3db7283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:56.976669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5b82c-f357-4b0d-9042-e59fa3db7283 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:56.982073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 382/778] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 13:59:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:56.983595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee083-c382-4856-8fa3-d8339ed78a94 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:56.983903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee083-c382-4856-8fa3-d8339ed78a94 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:56.984176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee083-c382-4856-8fa3-d8339ed78a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.004425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b2cee083-c382-4856-8fa3-d8339ed78a94 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:57.005896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b2cee083-c382-4856-8fa3-d8339ed78a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:57.081483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf085653-9e2a-4ed2-9e33-eca033abbb4e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.101071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:57.110187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 383/779] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 13:59:57.111897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee083-c382-4856-8fa3-d8339ed78a94 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.143264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.143887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.144169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.144501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.145212 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] RBAC: Adding query filter params (interface=public, service_id=c0f34289396f40e08a3b4f14bca0a3d4, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:57.145524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:57.147038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:57.165171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 397/780] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 13:59:56 2022] GET /identity/v3/endpoints?service_id=c0f34289396f40e08a3b4f14bca0a3d4&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:57.172082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.188088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:57.235491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee083-c382-4856-8fa3-d8339ed78a94 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.278842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.280085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.280882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.281518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.282461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] RBAC: Adding query filter params (type=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:57.283115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:57.285278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:57.291837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 384/781] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:57.293114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.330415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c253961-1d0e-43f9-b97d-06f5660ca396 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.365797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.366404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.366680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.366952 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.367546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:57.368894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:57.381757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 398/782] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:57.381757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb66df5-20c2-4831-98ce-0d4dd61af853 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.460429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:57.482217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:57.496054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.496705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.496982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.497255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.510938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:57.516549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:57.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 385/783] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:57.518840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.546836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd657-5f32-4a27-8a25-b2c5869ce338 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.600687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.601303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.601581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.601900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.602609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] RBAC: Adding query filter params (interface=public, service_id=7165cbdf67984e2db62b4e194546872d, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:57.602914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:57.604297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:57.614711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 399/784] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/endpoints?service_id=7165cbdf67984e2db62b4e194546872d&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:57.614711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.665705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6147db-41c2-4907-9b9a-ca86373f58b9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.718311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.719353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.719906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.720366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.734312 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:57.738950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d2173-079a-4162-ac75-274e2a076fed None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:57.740190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:57.741118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 13:59:57.742021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:57.744325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 386/785] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:57.744325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.759684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:57.800148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.801414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.801916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.802614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:57.803041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.805363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:57.805984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:57.808291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:57.822038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 400/786] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:57.823876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.870402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d73733-5031-4937-83a7-849ffc8ff2c0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:57.910152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:57.910785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:57.911064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:57.911335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:57.911940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:57.913369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:57.919095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 387/787] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:57.919935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:57.962276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b4b69-9c7a-4b82-8a98-47fe427bca52 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:58.030470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463d95e-377f-4adf-b21d-f4a7394d754c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:58.032227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:58.033028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.033315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.033598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.034302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:58.034662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:58.036053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:58.040907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:58.064125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 401/788] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:58.069466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:58.069933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:58.070141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.070420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.070690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.088244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:58.089879 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:58.136965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:58.242120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:58.243298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:58.244353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:58.244353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:58.244353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:58.244353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 388/789] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:58.245615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46318-218e-4ff3-8100-46d243255162 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:58.251227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:58.294149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:58.295062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.295759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.296440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.314653 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:58.319107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:58.341474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:58.342687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:58.343583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:58.343583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:58.343583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:58.348686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 402/790] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:58.354529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:58.414756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3b0bb-5bb0-4324-bd77-7c13bc961604 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:58.495452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:58.496074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.496351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.496617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.497274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:58.497577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:58.499568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:58.528318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 389/791] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:58.534708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:58.558622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:58.561859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:58.591889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7625b2-a9e6-4e53-999f-afc5a41e607c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:58.643876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221bf4f-268e-4357-9a11-4d72ae06c357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:58.644479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221bf4f-268e-4357-9a11-4d72ae06c357 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.644756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221bf4f-268e-4357-9a11-4d72ae06c357 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.645023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221bf4f-268e-4357-9a11-4d72ae06c357 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.645858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221bf4f-268e-4357-9a11-4d72ae06c357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:58.646263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221bf4f-268e-4357-9a11-4d72ae06c357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:58.647607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221bf4f-268e-4357-9a11-4d72ae06c357 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:58.655277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed2265-a716-4902-912d-16a6b4bc68fb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:58.661831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 403/792] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:58.663295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540d0f3-d8ff-4359-91f7-a2e964f8f24e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.663584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540d0f3-d8ff-4359-91f7-a2e964f8f24e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.663858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540d0f3-d8ff-4359-91f7-a2e964f8f24e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.669348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 404/793] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 13:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 13:59:58.670251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540d0f3-d8ff-4359-91f7-a2e964f8f24e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:58.691692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f7ce5-8d4c-42f1-95a7-11df8e65190c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:58.692695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f7ce5-8d4c-42f1-95a7-11df8e65190c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.693170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f7ce5-8d4c-42f1-95a7-11df8e65190c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.693639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f7ce5-8d4c-42f1-95a7-11df8e65190c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.701504 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f7ce5-8d4c-42f1-95a7-11df8e65190c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:58.703818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f7ce5-8d4c-42f1-95a7-11df8e65190c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-126f7ce5-8d4c-42f1-95a7-11df8e65190c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:58.716090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:58.717053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 13:59:58.718262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:58.720913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 390/794] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:58.726758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec0bb5-f839-4973-a202-b523d1ac5f88 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.727295 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec0bb5-f839-4973-a202-b523d1ac5f88 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.727778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec0bb5-f839-4973-a202-b523d1ac5f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.746409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a0ec0bb5-f839-4973-a202-b523d1ac5f88 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 13:59:58.748329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a0ec0bb5-f839-4973-a202-b523d1ac5f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 13:59:58.779126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540d0f3-d8ff-4359-91f7-a2e964f8f24e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:58.816256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:58.816793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:58.817068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:58.817337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:58.821042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 391/795] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 13:59:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 13:59:58.822164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec0bb5-f839-4973-a202-b523d1ac5f88 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:58.833786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:58.835552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:58.840772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 13:59:58.858512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:58.859511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 13:59:58.860472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:58.868064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 405/796] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 13:59:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:58.873677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:58.884527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:58.983173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec0bb5-f839-4973-a202-b523d1ac5f88 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.022563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.023598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.024493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.024965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.025916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:59.026421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.028400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.049095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 392/797] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 13 13:59:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:59.054676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.082510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5688f-9268-49e6-ad49-f7170d8d850e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.139690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.141158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.141686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.142191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.143125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:59.143624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.145418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.158463 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:59.167036 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 406/798] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 13 13:59:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 13:59:59.168247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.193224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3cd25-ddba-4843-8f6f-e147228b4c5b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.227925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73393013-0547-4173-84a3-d1516848f930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.228554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73393013-0547-4173-84a3-d1516848f930 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.228876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73393013-0547-4173-84a3-d1516848f930 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.229145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73393013-0547-4173-84a3-d1516848f930 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.239787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:59.239959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73393013-0547-4173-84a3-d1516848f930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.241276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73393013-0547-4173-84a3-d1516848f930 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-73393013-0547-4173-84a3-d1516848f930 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:59.256609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:59.257599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:59.258513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:59.258513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:59.258513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:59.262084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 393/799] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 13:59:59.262900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73393013-0547-4173-84a3-d1516848f930 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.285229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452253-b3b6-4b01-95a7-273145ff341b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.323134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.324203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.324709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.325188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.336727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 13:59:59.341963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:59.343140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 13:59:59.344107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 13:59:59.346604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 407/800] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:59.347593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.373483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73393013-0547-4173-84a3-d1516848f930 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.394189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:59.409188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.409862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.410142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.410411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.422421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.423834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 13:59:59.439117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:59.440118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 13:59:59.441007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 13:59:59.441007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 13:59:59.441007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:59.441007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 394/801] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:59.441658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.466230 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde544c3-7df3-48ea-8f8f-b7656e6de409 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.498154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:59.505374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.506388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.506965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.507432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.508327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:59.508826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.510531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.523767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 408/802] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:59.525170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.566258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d5cc-c531-49eb-b987-83e663164bfb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.602929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.604042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.604585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.605070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.605965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:59.606467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.608746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.623402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 395/803] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:59.625001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.663708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:59.737478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63780e10-3a84-4dcf-b1d4-642c8736adc7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.766408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb988959-e762-4c9d-9734-6c0d378e3df7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.776912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.778718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.779225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.780331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.789097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.791044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 13:59:59.805439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 13:59:59.806452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 13:59:59.807353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 396/804] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 13:59:59.813357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.815650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.817434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.817951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.818471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.819363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:59.819862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.821901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.837548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 409/805] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 13:59:59.838709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 13:59:59.895369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:59.928581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 13:59:59.937349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a345e-6f37-444e-97ba-008cd47b6f61 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.954290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54003108-718b-4f72-89f0-b7f663bd1d3e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 13:59:59.972711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.973339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.973614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.973943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.985711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 13:59:59.987156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 13:59:59.994721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56686d-4c36-4de3-948b-cf99857fcf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 13:59:59.996007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56686d-4c36-4de3-948b-cf99857fcf7e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 13:59:59.996702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56686d-4c36-4de3-948b-cf99857fcf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 13:59:59.997171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56686d-4c36-4de3-948b-cf99857fcf7e None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 13:59:59.998079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56686d-4c36-4de3-948b-cf99857fcf7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 13:59:59.998606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56686d-4c36-4de3-948b-cf99857fcf7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.000187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56686d-4c36-4de3-948b-cf99857fcf7e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:00.001776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:00.002706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:00.003595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:00.003595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:00.003595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:00.003595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 397/806] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:00.003831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.018997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 410/807] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 13:59:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:00.020044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f287-5f11-492b-bfc4-6a7d3e4550b1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.020401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f287-5f11-492b-bfc4-6a7d3e4550b1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.020743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f287-5f11-492b-bfc4-6a7d3e4550b1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.022301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 411/808] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:00.023114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3f287-5f11-492b-bfc4-6a7d3e4550b1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.116134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830d193-4b7f-4625-af8f-b62ebe27765b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.128721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3f287-5f11-492b-bfc4-6a7d3e4550b1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.131301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:00.157508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34083c4-a7ac-4878-a344-a7365d4191b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.158637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34083c4-a7ac-4878-a344-a7365d4191b8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.158994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34083c4-a7ac-4878-a344-a7365d4191b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.159275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34083c4-a7ac-4878-a344-a7365d4191b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.166142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.166627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34083c4-a7ac-4878-a344-a7365d4191b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.166785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.167053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.167294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.167787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34083c4-a7ac-4878-a344-a7365d4191b8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.167963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:00.168228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.169582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d34083c4-a7ac-4878-a344-a7365d4191b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:00.179702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:00.180605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:00.181534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 398/809] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:00.182636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d4f1c-55b9-4f0d-a550-f62d5ec31b35 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.182636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d4f1c-55b9-4f0d-a550-f62d5ec31b35 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.182636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d4f1c-55b9-4f0d-a550-f62d5ec31b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.192467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 412/810] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:00.192467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.200244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:00.206727 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-952d4f1c-55b9-4f0d-a550-f62d5ec31b35 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:00.208142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-952d4f1c-55b9-4f0d-a550-f62d5ec31b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:00.285999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 399/811] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:00.285999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d4f1c-55b9-4f0d-a550-f62d5ec31b35 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.312079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f701b-9568-4279-8765-b12f9a3a259f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.350870 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.351530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.351806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.352073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/15e78b0e48a141edad05f3c064ac04f7/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.354124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=15e78b0e48a141edad05f3c064ac04f7, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.355832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.372841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 413/812] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:00:00 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/15e78b0e48a141edad05f3c064ac04f7/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:00:00.374680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.412403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:00.423299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d4f1c-55b9-4f0d-a550-f62d5ec31b35 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.468010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b09890-b177-4a24-9326-54f3cad43194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.469044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09890-b177-4a24-9326-54f3cad43194 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.469539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09890-b177-4a24-9326-54f3cad43194 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.470031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09890-b177-4a24-9326-54f3cad43194 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.471027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b09890-b177-4a24-9326-54f3cad43194 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:00.471552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b09890-b177-4a24-9326-54f3cad43194 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.473226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b09890-b177-4a24-9326-54f3cad43194 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.477056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:00.495815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 400/813] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:00.495815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a469a-55eb-4ecc-870d-7ce899c75ee0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.495815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a469a-55eb-4ecc-870d-7ce899c75ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.495815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a469a-55eb-4ecc-870d-7ce899c75ee0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.495815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 401/814] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:00.500634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a469a-55eb-4ecc-870d-7ce899c75ee0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.531293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35adf5a-2b19-4862-a9bd-a0e9e58a6dba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.580752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf357a6f-8b1a-44a0-8847-63d3432a5a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.581402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf357a6f-8b1a-44a0-8847-63d3432a5a46 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.581768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf357a6f-8b1a-44a0-8847-63d3432a5a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.582042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf357a6f-8b1a-44a0-8847-63d3432a5a46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.590096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf357a6f-8b1a-44a0-8847-63d3432a5a46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.591543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf357a6f-8b1a-44a0-8847-63d3432a5a46 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cf357a6f-8b1a-44a0-8847-63d3432a5a46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:00.603850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:00.604851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:00.605846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 414/815] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:00.606458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1ff48-cbe4-4783-9ee8-3f57159ddf52 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.607820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1ff48-cbe4-4783-9ee8-3f57159ddf52 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.607820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1ff48-cbe4-4783-9ee8-3f57159ddf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.628907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cce1ff48-cbe4-4783-9ee8-3f57159ddf52 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:00.630137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cce1ff48-cbe4-4783-9ee8-3f57159ddf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:00.641147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a469a-55eb-4ecc-870d-7ce899c75ee0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.681586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.681919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.682190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.682467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/929edc18c5874105a349dcc2b4cad14c/roles/adad88d8a75747a381a2b923728d27f8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.698118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=929edc18c5874105a349dcc2b4cad14c, role_id=adad88d8a75747a381a2b923728d27f8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.699349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.702652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 415/816] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:00.703437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1ff48-cbe4-4783-9ee8-3f57159ddf52 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.720776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:00.726694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 402/817] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:00:00 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/929edc18c5874105a349dcc2b4cad14c/roles/adad88d8a75747a381a2b923728d27f8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:00:00.728350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.783169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:00.831547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1ff48-cbe4-4783-9ee8-3f57159ddf52 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.841596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4dbf2-0993-4eb1-93d3-03f66ed102a1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:00.879680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.880747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.881446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.883243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.884154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.886589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.898200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:00.898827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:00.899116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:00.899379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:00.900142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:00.900452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:00.906137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:00.907232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 14:00:00.908112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:00.909535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:00.911402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 416/818] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:00.918145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:00.947945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 403/819] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:00.954360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.054864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:01.086270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fa09b-edfa-44e5-b316-4adb71e20db0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.089956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fbd03-a548-4ae4-9c44-69f3b9f73d10 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.099499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:01.130296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.130887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.131233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.131538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.132249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:01.132574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.134021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.139315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 417/820] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:01.140191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.142020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.142853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.143346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.143816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.232945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.235120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.251394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd93c3-8018-4911-8a7d-bcec0526de57 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.293793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.294532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.295045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.295342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] PATH_INFO: `/v3/services/network` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.296333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.297707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:01.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:01.311998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:01.312848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 418/821] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:01 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:01.314296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.326467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:01.366032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 404/822] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:01.367247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.370833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:01.434304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d1884-70d7-448a-90e7-b1407a904204 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.473237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.474138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.474624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.475138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.475900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:01.476375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.477908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.478166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71cb93-7762-43c0-a24f-dc84ac5ba683 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.483850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 419/823] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Sep 13 14:00:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:01.484817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.529508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.530602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.530848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.531128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.531766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] RBAC: Adding query filter params (name=network) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:01.532071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.533555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.541883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 405/824] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:01.542869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.571657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:01.607429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3fa71-26dc-4226-ab04-410dbfd4263c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.638741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:01.648653 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.649546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.650007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.650469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.663069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.664724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.678214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac75f78-ceea-4579-88b7-056e755a99fd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:01.680289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:01.681444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:01.682554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:01.682554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:01.682554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:01.684234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 420/825] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:01.685197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.728473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.729125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.729410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.729684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.730454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.731838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.760873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 406/826] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:01.766248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.850362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb62ce-3dc9-491f-afa1-c86f98280f5f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.897336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:01.946987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:01.947624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:01.947901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:01.948197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:01.948820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] RBAC: Adding query filter params (type=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:01.949124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:01.950592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:01.955287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa061-244c-4071-a309-8f4fa5e96b49 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:01.964648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 421/827] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:01.970617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:01.984660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:02.002975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:02.004030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:02.004712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:02.005166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:02.006026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:02.006508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:02.008080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:02.024600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 407/828] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:02.032739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:02.120835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5195dbb-9474-4aac-ad46-130837d1b84e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:02.169963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0520ac-8735-4063-a129-d19ba6a9c19b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:02.176651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:02.177262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:02.177535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:02.178253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:02.178930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:02.180677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:02.210950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:02.211583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:02.211701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 422/829] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:01 2022] POST /identity/v3/endpoints => generated 337 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:00:02.211866 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:02.212142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:02.222029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:02.223945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:02.237181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:02.238290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:02.239188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:02.241719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 408/830] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:02.250961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:02.316465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:02.415575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d04e2-e42e-4081-a335-5370c4f864aa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:02.460478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36f22b-7b6f-44e8-a9d2-93c544a5590f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:02.461433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36f22b-7b6f-44e8-a9d2-93c544a5590f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:02.461963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36f22b-7b6f-44e8-a9d2-93c544a5590f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:02.462498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36f22b-7b6f-44e8-a9d2-93c544a5590f None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:02.463394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36f22b-7b6f-44e8-a9d2-93c544a5590f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:02.463906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36f22b-7b6f-44e8-a9d2-93c544a5590f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:02.465536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36f22b-7b6f-44e8-a9d2-93c544a5590f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:02.485417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 423/831] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:02.494891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:02.569273 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:02.640045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90388bfe-dc4c-40f3-b30d-b04279343d84 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:02.676254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201552e-7967-4f89-9b50-36b3e21122f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:02.676867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201552e-7967-4f89-9b50-36b3e21122f9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:02.677142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201552e-7967-4f89-9b50-36b3e21122f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:02.677413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201552e-7967-4f89-9b50-36b3e21122f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:02.678163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201552e-7967-4f89-9b50-36b3e21122f9 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=6a4ea803f6e44648bb217abd00e11c3c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:02.678468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201552e-7967-4f89-9b50-36b3e21122f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:02.679894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201552e-7967-4f89-9b50-36b3e21122f9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:02.693918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 409/832] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:02 2022] GET /identity/v3/role_assignments?user.id=6a4ea803f6e44648bb217abd00e11c3c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:02.741584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5283e-ed18-4f77-bd17-6be0ea6db2d5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:02.742275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5283e-ed18-4f77-bd17-6be0ea6db2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:02.742774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5283e-ed18-4f77-bd17-6be0ea6db2d5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:02.744698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 424/833] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:02.751536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6918-c40e-4d93-807f-57b8fd4f6074 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:02.752212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6918-c40e-4d93-807f-57b8fd4f6074 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:02.752701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6918-c40e-4d93-807f-57b8fd4f6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:02.772547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-07de6918-c40e-4d93-807f-57b8fd4f6074 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:02.774173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07de6918-c40e-4d93-807f-57b8fd4f6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:02.851862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 410/834] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:03.128134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5283e-ed18-4f77-bd17-6be0ea6db2d5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:03.235944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:03.243989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5283e-ed18-4f77-bd17-6be0ea6db2d5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:03.280761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:03.281600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:03.281770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:03.285279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:03.285389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcab91-0cae-43b9-9fd9-cd351f7227ab None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:03.285389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcab91-0cae-43b9-9fd9-cd351f7227ab None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:03.285389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcab91-0cae-43b9-9fd9-cd351f7227ab None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:03.285389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 411/835] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:03.290945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05447d20-67ed-488b-a4b2-530e32f3ac39 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:03.291499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05447d20-67ed-488b-a4b2-530e32f3ac39 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:03.291968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05447d20-67ed-488b-a4b2-530e32f3ac39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:03.297294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:03.302320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:03.303233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:03.304083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 425/836] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:03.309410 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:03.315862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-05447d20-67ed-488b-a4b2-530e32f3ac39 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:03.317334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-05447d20-67ed-488b-a4b2-530e32f3ac39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:03.352822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:03.408221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 412/837] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:03.433105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f27d82-d9bb-444f-845b-e07ac94d17d9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:03.474364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:03.475017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:03.475294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:03.475564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:03.476225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:03.476530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:03.478058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:03.487050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 426/838] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:00:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:03.492632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05447d20-67ed-488b-a4b2-530e32f3ac39 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:03.602678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05447d20-67ed-488b-a4b2-530e32f3ac39 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:03.640345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:03.641401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:03.641936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:03.642429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:03.654606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:03.656706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:03.672528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:03.673509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:03.674469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:03.674469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:03.674469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:03.678138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 413/839] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:03.679916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:03.699648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:03.702897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:03.728118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:03.797497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd674a-7e94-4c4b-84d5-be50e9c063c4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:03.824251 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62c8e7-5116-48cc-b9f2-7165d9283ab9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:03.840721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:03.841980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:03.842843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:03.843337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:03.844336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:03.845111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:03.846720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:03.862855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:03.863406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 427/840] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:03.863514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:03.863750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:03.864020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:03.869865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:03.881776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:03.888558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:03.890352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:03.896468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 414/841] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 14:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:03.898652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:03.962250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:03.988662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926eff05-8e12-4860-b603-49261c8ccacb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:03.992348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:04.016383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f97cfa-8299-4f25-a30c-9ca47d853eb9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.029429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.030494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.030976 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.031541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.039583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.041024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:04.059477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:04.060461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:04.062093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 428/842] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:00:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:04.066809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.067141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.067974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.068456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.068927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.069870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:04.070250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.071699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.084950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 415/843] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 14:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:04.086237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.175578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c12389-8c45-41f5-a48c-19aa82b73b59 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.211175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64bec5-a256-4235-8391-ce94e5a3e3ef None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.213006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.213554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.213862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.214141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.214868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:04.215184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.216715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.220179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:04.240689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 429/844] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:04.243042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.263918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d2bf1-40df-4f58-9e55-f2b5c1c0b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.264764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d2bf1-40df-4f58-9e55-f2b5c1c0b849 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.265039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d2bf1-40df-4f58-9e55-f2b5c1c0b849 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.265309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d2bf1-40df-4f58-9e55-f2b5c1c0b849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.278710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d2bf1-40df-4f58-9e55-f2b5c1c0b849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.280715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d2bf1-40df-4f58-9e55-f2b5c1c0b849 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.282757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8c8d2bf1-40df-4f58-9e55-f2b5c1c0b849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:04.298236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:04.299839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:04.300964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:04.300964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:04.300964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:04.302367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 416/845] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:04.303849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635795ce-8e9e-4f5c-b756-e2afee4bb16b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.304308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635795ce-8e9e-4f5c-b756-e2afee4bb16b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.304754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635795ce-8e9e-4f5c-b756-e2afee4bb16b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.306179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 417/846] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:04.307568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635795ce-8e9e-4f5c-b756-e2afee4bb16b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.362111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476563f-09e3-4549-85a5-7cdca67a3127 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.400714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.401268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.401547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.401855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.423080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635795ce-8e9e-4f5c-b756-e2afee4bb16b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.462370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1145793-c051-4bb5-8e2d-526d5b3ca871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.462993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1145793-c051-4bb5-8e2d-526d5b3ca871 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.463356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1145793-c051-4bb5-8e2d-526d5b3ca871 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.463579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1145793-c051-4bb5-8e2d-526d5b3ca871 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.464219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1145793-c051-4bb5-8e2d-526d5b3ca871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:04.464521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1145793-c051-4bb5-8e2d-526d5b3ca871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.465913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1145793-c051-4bb5-8e2d-526d5b3ca871 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.469945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:04.478302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.479732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.483458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 418/847] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:04.484703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d7004-fdc2-435c-91b0-ffecdd12e52b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.484991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d7004-fdc2-435c-91b0-ffecdd12e52b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.485266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d7004-fdc2-435c-91b0-ffecdd12e52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.505588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-234d7004-fdc2-435c-91b0-ffecdd12e52b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:04.506835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-234d7004-fdc2-435c-91b0-ffecdd12e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:04.572309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:04.597392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 430/848] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:04.597586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 419/849] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:04.597680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.599118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d7004-fdc2-435c-91b0-ffecdd12e52b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.715178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d7004-fdc2-435c-91b0-ffecdd12e52b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.722192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:04.729688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398a18-5a56-4e9d-8b02-71665d759b84 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.764910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.765527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.765916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.766191 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.767422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.768262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.768753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.769215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.777116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.778614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.779362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.780853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:04.792163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:04.793100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:04.794077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:04.796286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:04.797540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:04.798611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:04.798611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:04.798611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:04.799294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 431/850] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:04.801658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.802097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 420/851] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:04.807244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.852319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:04.913356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4b20-9aaf-40c3-a1e3-2368abff2ea6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.933099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610170c9-c706-4b86-9e5c-c34cf2ec98a6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:04.951007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.951983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.952555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.952679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.953315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:04.953624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.955086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.972749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:04.973282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:04.973557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:04.978400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:04.978539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 432/852] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:04.978622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:04.978707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.978780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:04.978780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:04.993718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 421/853] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:04.993718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:04.998017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.086841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f6afd-3087-49c3-a859-33fccce06b53 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.087610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.094198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4675e2-cc7b-4590-8493-809ce34abf7b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.121591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.122255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.122539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.122815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.123421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.125141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.127670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b487b-b609-40bf-872b-39fe6e15dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.128365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b487b-b609-40bf-872b-39fe6e15dce3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.128599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b487b-b609-40bf-872b-39fe6e15dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.128862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b487b-b609-40bf-872b-39fe6e15dce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:05.134646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:05.135637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:05.136488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:05.140728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 433/854] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:05.141820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.197550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b487b-b609-40bf-872b-39fe6e15dce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.199006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b487b-b609-40bf-872b-39fe6e15dce3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.217045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.242404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe1ec2-8259-49fb-9315-d0a360f1a710 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.274719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.275333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.275604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.275872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.282794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.284214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.290362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:05.295905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:05.297111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:05.298324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:05.298963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 422/855] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:04 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:05.299370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04765445-ef07-4328-aac1-d6953288c47c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.299370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04765445-ef07-4328-aac1-d6953288c47c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.299610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04765445-ef07-4328-aac1-d6953288c47c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.300766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 423/856] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:05.302073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 434/857] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:05.302880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.305692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04765445-ef07-4328-aac1-d6953288c47c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.402300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51aeaf-68f4-483c-8bef-784ee594a9c2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.404603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04765445-ef07-4328-aac1-d6953288c47c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.435906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.436579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.436890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.437160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.438579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489555b-c741-433b-b2b2-16952fa1896f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.439174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489555b-c741-433b-b2b2-16952fa1896f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.439455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489555b-c741-433b-b2b2-16952fa1896f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.439728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489555b-c741-433b-b2b2-16952fa1896f None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.439832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.440497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489555b-c741-433b-b2b2-16952fa1896f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:05.440655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489555b-c741-433b-b2b2-16952fa1896f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.442049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489555b-c741-433b-b2b2-16952fa1896f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.446406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 424/858] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:05.447526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0018-382d-4701-a7db-70ae80cd9280 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.447804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0018-382d-4701-a7db-70ae80cd9280 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.448072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c0018-382d-4701-a7db-70ae80cd9280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.454058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.456422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.465839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-372c0018-382d-4701-a7db-70ae80cd9280 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:05.467016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-372c0018-382d-4701-a7db-70ae80cd9280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:05.471821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:05.473076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:05.474240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:05.474240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:05.474240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:05.474240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 435/859] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:05.474240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.523276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.535969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 425/860] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:05.536803 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c0018-382d-4701-a7db-70ae80cd9280 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.582146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91aae5-a9a0-41c1-b6a8-8ae07a310e55 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.618243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.619601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.619885 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.620160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.620848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:05.621157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.622619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.636774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 436/861] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:05.637560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.654569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c0018-382d-4701-a7db-70ae80cd9280 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.689706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.694531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.695072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.695388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.695633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.696263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:05.696549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.698429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.713760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 426/862] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:05.713760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.765151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262187-7406-4af5-9908-9baa58d980d5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.783109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.804272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.805057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.805404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.806113 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.806635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:05.806939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.808856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.821511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 437/863] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:05.822457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.843839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03698fb-843c-4dda-b37e-2cb4e7c51376 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.886812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:05.887642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:05.887975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:05.888269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:05.889202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=15e78b0e48a141edad05f3c064ac04f7) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:05.889507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:05.890974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:05.905922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 427/864] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/role_assignments?user.id=15e78b0e48a141edad05f3c064ac04f7&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:05.906973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:05.947649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a68f9-8f4f-47bf-8972-e37973d0d51d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:05.962650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:05.999750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:06.000397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:06.000676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:06.000947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:06.002145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:06.003841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:06.020041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:06.021078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:06.022031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 438/865] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:06.029793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:06.058341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef5844-b51a-469e-8b7f-fa7eb1b6328d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:06.082962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:06.103680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:06.104311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:06.104586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:06.104854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:06.113139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:06.114642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:06.129687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:06.130740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:06.131655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 428/866] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:06.137471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:06.178361 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823bf3e-99ee-4e7d-b70f-6b41897bba1d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:06.217156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:06.219071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:06.219612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:06.220141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:06.221059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:06.221563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:06.223236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:06.228822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 439/867] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:06.235000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:06.258931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9973c8-f74e-4a0b-9ce7-7e3fab982bb7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:06.275400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:06.303465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:06.304077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:06.304349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:06.304621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:06.305310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:06.305626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:06.307305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:06.327531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 429/868] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:06.328691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:06.355254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadace59-e395-4990-ab64-9cedacf00f63 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:06.374921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:06.397198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36743618-b7e7-4a27-aa72-1bd2a8940f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:06.398398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36743618-b7e7-4a27-aa72-1bd2a8940f01 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:06.398955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36743618-b7e7-4a27-aa72-1bd2a8940f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:06.399443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36743618-b7e7-4a27-aa72-1bd2a8940f01 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:06.400792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36743618-b7e7-4a27-aa72-1bd2a8940f01 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:06.401578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36743618-b7e7-4a27-aa72-1bd2a8940f01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:06.403464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36743618-b7e7-4a27-aa72-1bd2a8940f01 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:06.414081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 440/869] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:06.446387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b331f-17bc-4a03-8e4d-b90d48e4e25f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:06.485604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03066f61-065e-4ec9-b602-e87777dba21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:06.486254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03066f61-065e-4ec9-b602-e87777dba21c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:06.486531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03066f61-065e-4ec9-b602-e87777dba21c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:06.486800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03066f61-065e-4ec9-b602-e87777dba21c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:06.487527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03066f61-065e-4ec9-b602-e87777dba21c None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=929edc18c5874105a349dcc2b4cad14c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:06.487833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03066f61-065e-4ec9-b602-e87777dba21c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:06.489261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03066f61-065e-4ec9-b602-e87777dba21c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:06.496313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 430/870] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:06 2022] GET /identity/v3/role_assignments?user.id=929edc18c5874105a349dcc2b4cad14c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:07.926827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c8e6-6c08-4f60-8cfb-12f89f061696 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:07.927320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c8e6-6c08-4f60-8cfb-12f89f061696 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:07.927320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31c8e6-6c08-4f60-8cfb-12f89f061696 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:07.928855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 441/871] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:07.934156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b26ad-9c72-44d4-ac73-8b18ece4ef46 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:07.934569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b26ad-9c72-44d4-ac73-8b18ece4ef46 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:07.934740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b26ad-9c72-44d4-ac73-8b18ece4ef46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:07.952598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3d6b26ad-9c72-44d4-ac73-8b18ece4ef46 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:07.953797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3d6b26ad-9c72-44d4-ac73-8b18ece4ef46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:08.020194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 431/872] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:08.182143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f90de-9153-4b2d-9f7f-b4d2a5bdc906 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.182316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f90de-9153-4b2d-9f7f-b4d2a5bdc906 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.182651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f90de-9153-4b2d-9f7f-b4d2a5bdc906 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.184305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 442/873] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:08.190415 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc8d16-4daf-434d-8321-249517a80551 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.190739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc8d16-4daf-434d-8321-249517a80551 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.190971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc8d16-4daf-434d-8321-249517a80551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.210090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-48fc8d16-4daf-434d-8321-249517a80551 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:08.211389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-48fc8d16-4daf-434d-8321-249517a80551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:08.237914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:08.285973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 432/874] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:08.290811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f90de-9153-4b2d-9f7f-b4d2a5bdc906 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:08.396471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:08.400595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f90de-9153-4b2d-9f7f-b4d2a5bdc906 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:08.438694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aa8e0-f47e-424c-a42c-ea76608d8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:08.439219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aa8e0-f47e-424c-a42c-ea76608d8793 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.439499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aa8e0-f47e-424c-a42c-ea76608d8793 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.439771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aa8e0-f47e-424c-a42c-ea76608d8793 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.440424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95aa8e0-f47e-424c-a42c-ea76608d8793 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:08.442491 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95aa8e0-f47e-424c-a42c-ea76608d8793 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:08.460560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 443/875] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:00:08 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:08.568348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc8d16-4daf-434d-8321-249517a80551 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:08.687626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc8d16-4daf-434d-8321-249517a80551 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:08.729387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e4e54-6c9c-4e7f-8051-568c2c63e92e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.729785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e4e54-6c9c-4e7f-8051-568c2c63e92e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.729986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e4e54-6c9c-4e7f-8051-568c2c63e92e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.731646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 444/876] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:08.737884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3861d2-702d-4673-a081-426e4213cca0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.738169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3861d2-702d-4673-a081-426e4213cca0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.738447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3861d2-702d-4673-a081-426e4213cca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.741257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:08.742183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.742619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.743043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.743778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:08.745283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:08.755336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:08.756661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:08.757809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:08.759517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0b3861d2-702d-4673-a081-426e4213cca0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:08.760438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 433/877] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:08.760858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b3861d2-702d-4673-a081-426e4213cca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:08.769012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:08.795101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:08.836310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 445/878] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:08.837648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc8032-b051-453e-a128-2c04496ded6a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.837967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc8032-b051-453e-a128-2c04496ded6a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.838241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc8032-b051-453e-a128-2c04496ded6a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.839628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 446/879] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:08.845401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f388-1ddc-4081-832e-9147a2c537d8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.845966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f388-1ddc-4081-832e-9147a2c537d8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.846459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f388-1ddc-4081-832e-9147a2c537d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.865658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-22c0f388-1ddc-4081-832e-9147a2c537d8 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:08.867199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-22c0f388-1ddc-4081-832e-9147a2c537d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:08.885624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bbb3eb-d6af-4c16-b6ff-fd6f265f3592 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:08.922106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:08.922743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:08.923019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:08.923287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:08.923910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:08.924209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:08.925556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:08.933838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 434/880] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:08.937045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:08.965791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 447/881] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:09.031937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:09.047626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1198839a-803a-4d9b-9666-ae3d2b111b07 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:09.081839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:09.082440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:09.082714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:09.082984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:09.083603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:09.083901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:09.085251 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:09.093861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 435/882] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:09.131445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f388-1ddc-4081-832e-9147a2c537d8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:09.157468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:09.247876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f388-1ddc-4081-832e-9147a2c537d8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:09.248969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:09.287750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:09.288300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:09.288596 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:09.288875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:09.289557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:09.291039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:09.309130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 448/883] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:00:09 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:09.373190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5950b-edb6-48c5-bfb0-49a438be04f2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:09.419239 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:09.420395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:09.420997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:09.421483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:09.433607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:09.440158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:09.441213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:09.442114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:09.445001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 436/884] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 13 14:00:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:09.454964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:09.555621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:09.577541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5d71-5018-483c-8f73-1722fe9d1f0a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:09.611607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:09.612214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:09.612487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:09.612758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:09.613421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:09.613719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:09.615108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:09.624173 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 449/885] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 13 14:00:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:09.629801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:09.640401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:09.742985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be2d72-2b26-46fb-a843-c438c73365af None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:09.778732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:09.779352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:09.779732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:09.780148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:09.791789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:09.793219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:09.808537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:09.809563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:09.810680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:09.810680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:09.810680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:09.810680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 437/886] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:09.815817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:09.925502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097052dd-2c72-4d56-b9cc-c06880a7f1de None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:09.961824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:09.962857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:09.963387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:09.963876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:09.964728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:09.965252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:09.966838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:09.980979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 450/887] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:09.986515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:10.047513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:10.093404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da26bc-318b-4b05-ba9f-c77ec31cd94e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:10.131958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:10.132630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:10.132960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:10.133304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:10.142413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:10.143846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:10.156220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:10.157482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:10.158512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 438/888] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:00:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:10.163322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:10.168286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:10.293652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea7b51-cf34-407a-854c-965268ddab88 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:10.331742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:10.332777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:10.333275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:10.333829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:10.334824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:10.335421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:10.337221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:10.362811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 451/889] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 13 14:00:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:10.371090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:10.478479 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61a815-dbac-4fb2-9bc4-8fb9f0ff5eee None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:10.495856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:10.515615 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47546087-183a-4e1a-b997-02212c4254db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:10.516220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47546087-183a-4e1a-b997-02212c4254db None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:10.516538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47546087-183a-4e1a-b997-02212c4254db None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:10.516813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47546087-183a-4e1a-b997-02212c4254db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:10.587997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47546087-183a-4e1a-b997-02212c4254db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:10.589449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47546087-183a-4e1a-b997-02212c4254db None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:10.696950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 439/890] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:10 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:10.702656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:10.767423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:10.783222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158d0bd-b751-4444-a50a-f3db8787ae55 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:10.783856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158d0bd-b751-4444-a50a-f3db8787ae55 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:10.784342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158d0bd-b751-4444-a50a-f3db8787ae55 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:10.786299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 440/891] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:10.790892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466716d-6ecb-40df-97c6-03e2f0aab62d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:10.791488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466716d-6ecb-40df-97c6-03e2f0aab62d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:10.791969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466716d-6ecb-40df-97c6-03e2f0aab62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:10.811264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef789c-d274-4d33-98fe-3894d6affe41 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:10.814690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a466716d-6ecb-40df-97c6-03e2f0aab62d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:10.816120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a466716d-6ecb-40df-97c6-03e2f0aab62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:10.833578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:10.847560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:10.848198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:10.848570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:10.848914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:10.861244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:10.862854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:10.878824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:10.879856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:10.881635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:10.881635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:10.881635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:10.881635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 452/892] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:10.885558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:10.901991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 441/893] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:10.996591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11c063-4c5e-4143-8e3c-023d8a6a5ae6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.010787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:11.035417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.036332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.036753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.037165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.037994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:11.038451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.040057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.052818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 453/894] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:11.057981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466716d-6ecb-40df-97c6-03e2f0aab62d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:11.165971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466716d-6ecb-40df-97c6-03e2f0aab62d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.189255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:11.202707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.203621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.204086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.204646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.212141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.213578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:11.226826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:11.227800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:11.228681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 442/895] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:11.234244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:11.246138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:11.304862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d8665-23a7-4d3a-b4f2-31fb4079174a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.343327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.344283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.344758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.345337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.346223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.347808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:11.358528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:11.359639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:11.360498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:11.361250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557baed-f3df-43c0-b09c-838380b62a6f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.368245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 454/896] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:11.369372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:11.396651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.397396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.397592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.397871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.398578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:11.398887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.400761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.413778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:11.425000 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 443/897] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:11.426780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:11.487552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efcdfc-9594-484d-9db4-b95188351820 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.510752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:11.526152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3deaa-b0ce-4fb2-9024-6ae040400e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.526869 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3deaa-b0ce-4fb2-9024-6ae040400e9a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.527255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3deaa-b0ce-4fb2-9024-6ae040400e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.527630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3deaa-b0ce-4fb2-9024-6ae040400e9a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.528649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3deaa-b0ce-4fb2-9024-6ae040400e9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:11.529406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3deaa-b0ce-4fb2-9024-6ae040400e9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.531673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3deaa-b0ce-4fb2-9024-6ae040400e9a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.536883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 455/898] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:11.538091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083e5b-2aa6-4d73-ac87-c0b79a34c6d4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.538396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083e5b-2aa6-4d73-ac87-c0b79a34c6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.538665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083e5b-2aa6-4d73-ac87-c0b79a34c6d4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.540061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 456/899] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:11.546585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17083e5b-2aa6-4d73-ac87-c0b79a34c6d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:11.556804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a1da3-5337-4d7a-a282-051c4da25ac5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.599594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03906e2-3bee-41bb-9f2d-840383f344c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.600876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03906e2-3bee-41bb-9f2d-840383f344c0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.601358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03906e2-3bee-41bb-9f2d-840383f344c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.602038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03906e2-3bee-41bb-9f2d-840383f344c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.603347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03906e2-3bee-41bb-9f2d-840383f344c0 None admin] RBAC: Adding query filter params (role.id=143ac081ca204690a023a1a2c385de76, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=929edc18c5874105a349dcc2b4cad14c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:11.603887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03906e2-3bee-41bb-9f2d-840383f344c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.605550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03906e2-3bee-41bb-9f2d-840383f344c0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.614980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 444/900] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/role_assignments?user.id=929edc18c5874105a349dcc2b4cad14c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=143ac081ca204690a023a1a2c385de76 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:11.617103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12337a5-0029-428e-8a6b-8bdd4788041b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.617615 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12337a5-0029-428e-8a6b-8bdd4788041b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.618507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12337a5-0029-428e-8a6b-8bdd4788041b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.643421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a12337a5-0029-428e-8a6b-8bdd4788041b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:11.645819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a12337a5-0029-428e-8a6b-8bdd4788041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:11.667490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:11.672914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17083e5b-2aa6-4d73-ac87-c0b79a34c6d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.711169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c3c2b-4453-45d9-9ff9-de63529486fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.711877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3c2b-4453-45d9-9ff9-de63529486fc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.712229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3c2b-4453-45d9-9ff9-de63529486fc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.712575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3c2b-4453-45d9-9ff9-de63529486fc None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.713301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c3c2b-4453-45d9-9ff9-de63529486fc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:11.713676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c3c2b-4453-45d9-9ff9-de63529486fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.715185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c3c2b-4453-45d9-9ff9-de63529486fc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.725764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 457/901] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:11.725764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34228ada-d7fe-4714-9b4e-e2ff4361cd69 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.725764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34228ada-d7fe-4714-9b4e-e2ff4361cd69 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.725764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34228ada-d7fe-4714-9b4e-e2ff4361cd69 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.729900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 458/902] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:11.731080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34228ada-d7fe-4714-9b4e-e2ff4361cd69 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:11.747077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 445/903] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:11.748990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb887f9-a75b-489c-a8ec-c7b9f457d253 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.749357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb887f9-a75b-489c-a8ec-c7b9f457d253 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.749766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb887f9-a75b-489c-a8ec-c7b9f457d253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.772440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4fb887f9-a75b-489c-a8ec-c7b9f457d253 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:11.773909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4fb887f9-a75b-489c-a8ec-c7b9f457d253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:11.851059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 446/904] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:11.854114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:11.860009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34228ada-d7fe-4714-9b4e-e2ff4361cd69 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:11.897061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:11.898053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:11.898576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:11.899076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:11.900097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] RBAC: Adding query filter params (interface=public, service_id=1f41601d7396499f8cbfb2d873107079, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:11.900612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:11.902266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:11.909176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 459/905] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/endpoints?service_id=1f41601d7396499f8cbfb2d873107079&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:11.914383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb887f9-a75b-489c-a8ec-c7b9f457d253 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:12.019515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb887f9-a75b-489c-a8ec-c7b9f457d253 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:12.056009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:12.056651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:12.056926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:12.057197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:12.057820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:12.058272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:12.059341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:12.065830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 447/906] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 14:00:11 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:00:12.091118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:12.132629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:12.181380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bc18e-8b9e-48ad-8c57-0d359f0694b5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:12.206617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:12.228731 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:12.229347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:12.229620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:12.229937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:12.230671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:12.233419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:12.254422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 460/907] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:00:12 2022] POST /identity/v3/services => generated 241 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:12.263324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf6dcb-5b72-4895-89ed-e4a5b145aa08 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:12.317232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:12.318779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:12.318779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:12.318779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:12.319034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:12.320475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:12.331306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:12.332354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:12.333268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 448/908] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Sep 13 14:00:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:12.340754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:12.442666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:12.459394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82375f1-ffd4-472d-8ed4-16fed657d217 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:12.499940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:12.500554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:12.500826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:12.501092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:12.501715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:12.502040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:12.503396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:12.508263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 461/909] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:12.514105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:12.630559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ea25a-116d-4464-ac4a-8c1d0db297c5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:12.647059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:12.666662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:12.667268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:12.667545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:12.667814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:12.668438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:12.668735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:12.670141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:12.680068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 449/910] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:12.681392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:12.784460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91af54-d194-49d0-99ea-e5e0067ce070 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:12.820876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946b3da-560e-454c-bd1c-c8f21ba24088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:12.821491 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b3da-560e-454c-bd1c-c8f21ba24088 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:12.821802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b3da-560e-454c-bd1c-c8f21ba24088 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:12.822080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946b3da-560e-454c-bd1c-c8f21ba24088 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:12.822775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b3da-560e-454c-bd1c-c8f21ba24088 None admin] RBAC: Adding query filter params (interface=public, service_id=a95eca1ab10b4687b45b56de06c5279f, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:12.823086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b3da-560e-454c-bd1c-c8f21ba24088 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:12.824597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946b3da-560e-454c-bd1c-c8f21ba24088 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:12.831075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 462/911] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 14:00:12 2022] GET /identity/v3/endpoints?service_id=a95eca1ab10b4687b45b56de06c5279f&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:12.836480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:12.944894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc90f-e99c-4cf8-9996-7da380c6bfc8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:12.980071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beabdce-4914-471c-b025-0142dde389af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:12.981097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beabdce-4914-471c-b025-0142dde389af None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:12.981577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beabdce-4914-471c-b025-0142dde389af None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:12.982074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beabdce-4914-471c-b025-0142dde389af None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:12.983560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beabdce-4914-471c-b025-0142dde389af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:12.985235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beabdce-4914-471c-b025-0142dde389af None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:12.992115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 450/912] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 14:00:12 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:00:13.895665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53be2c-b8b9-4465-9abf-07095a449109 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:13.896642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53be2c-b8b9-4465-9abf-07095a449109 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:13.897125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53be2c-b8b9-4465-9abf-07095a449109 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:13.898971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 463/913] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:13.905834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5352f75-2678-45de-bb69-00a7a5d35893 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:13.906905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5352f75-2678-45de-bb69-00a7a5d35893 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:13.907383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5352f75-2678-45de-bb69-00a7a5d35893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:13.929408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b5352f75-2678-45de-bb69-00a7a5d35893 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:13.931002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b5352f75-2678-45de-bb69-00a7a5d35893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:13.975185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:14.004534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 451/914] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:14.282915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53be2c-b8b9-4465-9abf-07095a449109 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:14.306537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:14.397844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53be2c-b8b9-4465-9abf-07095a449109 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:14.433629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:14.434671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:14.435165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:14.435642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:14.447750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:14.453265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:14.454240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:14.455079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:14.457669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 464/915] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 13 14:00:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:14.463770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5352f75-2678-45de-bb69-00a7a5d35893 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:14.589845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5352f75-2678-45de-bb69-00a7a5d35893 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:14.626722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8a286-edc5-4712-ae9f-ccdc76d20999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:14.627363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8a286-edc5-4712-ae9f-ccdc76d20999 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:14.627641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8a286-edc5-4712-ae9f-ccdc76d20999 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:14.627969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8a286-edc5-4712-ae9f-ccdc76d20999 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:14.628702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8a286-edc5-4712-ae9f-ccdc76d20999 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:14.629011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8a286-edc5-4712-ae9f-ccdc76d20999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:14.630402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8a286-edc5-4712-ae9f-ccdc76d20999 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:14.640641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 452/916] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 13 14:00:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:14.646578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:14.662765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c72e3e-75d7-459a-aac8-4869d12ebc1d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:14.663068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c72e3e-75d7-459a-aac8-4869d12ebc1d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:14.663323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c72e3e-75d7-459a-aac8-4869d12ebc1d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:14.665470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 453/917] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:14.674947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022783fa-adb2-4f7c-b296-0f11a4f09cab None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:14.675462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022783fa-adb2-4f7c-b296-0f11a4f09cab None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:14.675958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022783fa-adb2-4f7c-b296-0f11a4f09cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:14.699835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-022783fa-adb2-4f7c-b296-0f11a4f09cab None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:14.701034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-022783fa-adb2-4f7c-b296-0f11a4f09cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:14.730835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:14.732833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:14.755661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a1915-9da5-4e5e-ba10-12bf4391d04e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:14.784645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 454/918] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:14.786320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ede9d-4bd9-48fe-a37d-91b028d67fb4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:14.786810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ede9d-4bd9-48fe-a37d-91b028d67fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:14.787318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ede9d-4bd9-48fe-a37d-91b028d67fb4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:14.792730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 455/919] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:14.794738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94f65-f5ee-4960-b972-201f42bc5574 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:14.795020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94f65-f5ee-4960-b972-201f42bc5574 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:14.795367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a94f65-f5ee-4960-b972-201f42bc5574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:14.813767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:14.814424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:14.814705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:14.814982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:14.818612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-49a94f65-f5ee-4960-b972-201f42bc5574 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:14.820015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-49a94f65-f5ee-4960-b972-201f42bc5574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:14.827514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:14.829395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:14.849285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:14.850323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:14.851283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:14.851283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:14.851283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:14.851283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 465/920] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:14.858997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:14.906201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 456/921] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:14.969745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fed7c6-7780-40c6-9c20-cd7ed57ba053 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:14.996408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.005463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.006099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.006378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.006651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.007298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:15.007598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.008959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.021756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 466/922] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:15.027127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a94f65-f5ee-4960-b972-201f42bc5574 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.086961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.138043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.146566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a94f65-f5ee-4960-b972-201f42bc5574 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.187542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.188592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.189387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.190096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.197583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc6b2a-8a7e-4abc-9848-20e1bff2aadc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.198733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.200403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:15.214478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:15.215723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:15.217147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:15.218015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 457/923] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:15.222719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.237783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.238406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.238686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.238957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.239552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.241779 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:15.251993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:15.253000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:15.253873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 467/924] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:15.254687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.307210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.330036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19ce76-807d-4f2b-acb5-21e54b95b1d0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.352983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41854e-ed52-45a0-96fc-bf69cac92f9a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.363211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab7ac1-e0c1-4d79-a3b1-03932a26c380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.363834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab7ac1-e0c1-4d79-a3b1-03932a26c380 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.364108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab7ac1-e0c1-4d79-a3b1-03932a26c380 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.364379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab7ac1-e0c1-4d79-a3b1-03932a26c380 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.364957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab7ac1-e0c1-4d79-a3b1-03932a26c380 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.366432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab7ac1-e0c1-4d79-a3b1-03932a26c380 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-34ab7ac1-e0c1-4d79-a3b1-03932a26c380 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:15.375765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:15.376776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:15.377621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 458/925] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:15.378484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6963afa-b28c-4a60-adf7-cb2b67f634f1 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.378484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6963afa-b28c-4a60-adf7-cb2b67f634f1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.378484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6963afa-b28c-4a60-adf7-cb2b67f634f1 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.379826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 459/926] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:15.380382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6963afa-b28c-4a60-adf7-cb2b67f634f1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.394102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.394636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.394913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.395179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.395897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:15.396196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.397611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.402607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.415848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 468/927] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:15.416654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.487025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6963afa-b28c-4a60-adf7-cb2b67f634f1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.512720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866f8d58-feab-43cf-a0a7-01097033983b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.522249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15012c60-7811-453a-923a-6ab5ef408fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.522859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15012c60-7811-453a-923a-6ab5ef408fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.523322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.523500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15012c60-7811-453a-923a-6ab5ef408fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.523766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15012c60-7811-453a-923a-6ab5ef408fe4 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.524675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15012c60-7811-453a-923a-6ab5ef408fe4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:15.524979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15012c60-7811-453a-923a-6ab5ef408fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.526375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15012c60-7811-453a-923a-6ab5ef408fe4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.532768 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 460/928] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:15.534115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d17f-daa4-4f09-8953-2b1d3c3ee865 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.534394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d17f-daa4-4f09-8953-2b1d3c3ee865 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.534666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d17f-daa4-4f09-8953-2b1d3c3ee865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.548973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.549608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.549918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.550191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.550808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:15.551109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.552460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.552904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ea05d17f-daa4-4f09-8953-2b1d3c3ee865 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:15.554114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ea05d17f-daa4-4f09-8953-2b1d3c3ee865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:15.557264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 469/929] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:15.557864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.621708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 461/930] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:15.622573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d17f-daa4-4f09-8953-2b1d3c3ee865 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.637996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.657186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34489c-c13e-4f83-8098-d1837756368c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.691148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.691773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.692047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.692314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.702267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.703696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:15.720656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:15.721616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:15.722581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:15.722581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:15.722581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:15.722581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 470/931] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:15.723291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.737899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d17f-daa4-4f09-8953-2b1d3c3ee865 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.750998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.774081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.774683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.774960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.775230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.776015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:15.776320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.777866 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.783579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 462/932] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:15.788917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.823812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cc684-e008-49eb-9fc0-6749bf97d66c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.858699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.859299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.859571 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.859833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.860459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:15.860754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.862132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.866899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 471/933] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:15.867667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.876071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.901707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ebf1d-4e63-4ebc-ac99-5ad4e5eaeee2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:15.937466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:15.938109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:15.938300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:15.938537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:15.939168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:15.939464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:15.940815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:15.952420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 463/934] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:15.953221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:15.963628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:15.974472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40c4ed-41b3-41f2-970f-7e766f098f64 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.007050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.007652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.007933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.008207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.008862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.010461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.031548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 472/935] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:16.032580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.074078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4370dbd-d4f1-4af8-8a4b-bc4e20a339c1 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.122974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.124193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.124494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.124772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.125494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] RBAC: Adding query filter params (interface=public, service_id=9aad3b2290c546ab88bfbc143b843fe9, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:16.125839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.127481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.129762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:16.137242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 464/936] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 14:00:15 2022] GET /identity/v3/endpoints?service_id=9aad3b2290c546ab88bfbc143b843fe9&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:16.143000 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.154449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79725-ac23-483b-8463-3d176e3c6a41 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.193591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.194178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.194460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.194760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.195323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.196706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:16.206273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:16.207470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:16.208748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 473/937] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Sep 13 14:00:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:16.209765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.223432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:16.278935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecd284-bc95-401b-91fe-2ef0e70588d5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.315111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.315741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.316016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.316281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.320706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85418c-6d3c-4c0e-9cfd-d274ed7a42bb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.323647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.324502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:16.336568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:16.338603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 465/938] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:16.340386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.365038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.365665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.365980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.366262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.366843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.368185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.374285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 474/939] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 14:00:16 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:00:16.375068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.423735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:16.472915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:16.475783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc660f7-70a7-4da5-8519-03e844ca7755 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.504420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5c6ac-d602-4688-8f9e-1d0c02af64bd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.519254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.519881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.520160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.520435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.521076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:16.521383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.523338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.530999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 466/940] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:16.541600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.561786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.561786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.561786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.561786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.565629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:16.565629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.565629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.579830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 475/941] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:16.585376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.687805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342d79f-4dcd-45a8-ae45-f750e36e52d5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.702352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b3956-0a1e-476e-988f-b6097b93e496 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.718059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:16.734374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.735167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.735448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.735973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.737022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:16.737597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.739006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.741627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5717df-64ee-43b0-919d-8e7a8fbf5d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.742313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5717df-64ee-43b0-919d-8e7a8fbf5d90 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.742593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5717df-64ee-43b0-919d-8e7a8fbf5d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.742867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5717df-64ee-43b0-919d-8e7a8fbf5d90 None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/929edc18c5874105a349dcc2b4cad14c/roles/143ac081ca204690a023a1a2c385de76` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.744518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 467/942] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:16.745041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5717df-64ee-43b0-919d-8e7a8fbf5d90 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=929edc18c5874105a349dcc2b4cad14c, role_id=143ac081ca204690a023a1a2c385de76)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.749768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5717df-64ee-43b0-919d-8e7a8fbf5d90 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.750687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:16.754197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:16.771282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 476/943] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:00:16 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/929edc18c5874105a349dcc2b4cad14c/roles/143ac081ca204690a023a1a2c385de76 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:00:16.876209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a6c19-7369-495a-bbc3-5ee9e883f75d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:16.915313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b6631-0c5f-414a-b863-7e48bc15ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:16.916369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6631-0c5f-414a-b863-7e48bc15ec42 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:16.916849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6631-0c5f-414a-b863-7e48bc15ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:16.917318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6631-0c5f-414a-b863-7e48bc15ec42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:16.918334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b6631-0c5f-414a-b863-7e48bc15ec42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:16.919967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b6631-0c5f-414a-b863-7e48bc15ec42 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:16.947789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 468/944] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:18.285480 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f957-26ef-41bf-9c07-fc93fc595b00 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:18.286057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f957-26ef-41bf-9c07-fc93fc595b00 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:18.286057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879f957-26ef-41bf-9c07-fc93fc595b00 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:18.287540 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 477/945] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:18.293330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca815f-e0f6-4a0c-8083-12728a38d7c0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:18.293842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca815f-e0f6-4a0c-8083-12728a38d7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:18.293947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca815f-e0f6-4a0c-8083-12728a38d7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:18.313684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d1ca815f-e0f6-4a0c-8083-12728a38d7c0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:18.314918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d1ca815f-e0f6-4a0c-8083-12728a38d7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:18.352791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:18.393461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 469/946] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:18.674769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879f957-26ef-41bf-9c07-fc93fc595b00 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:18.693222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3304b1-9fb6-4a7b-a55d-b70411188cdc None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:18.693548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3304b1-9fb6-4a7b-a55d-b70411188cdc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:18.693802 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3304b1-9fb6-4a7b-a55d-b70411188cdc None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:18.695244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 470/947] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:18.701009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820a2f-8a5c-4448-ad00-2411c5e5bb40 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:18.701323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820a2f-8a5c-4448-ad00-2411c5e5bb40 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:18.702193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820a2f-8a5c-4448-ad00-2411c5e5bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:18.728736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d2820a2f-8a5c-4448-ad00-2411c5e5bb40 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:18.731093 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d2820a2f-8a5c-4448-ad00-2411c5e5bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:18.792067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879f957-26ef-41bf-9c07-fc93fc595b00 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:18.829770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 471/948] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:18.838713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:18.839590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:18.840018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:18.840434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:18.841245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:18.843083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:18.846140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING keystone.server.flask.common [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 14:00:18.867546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.sql.core [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 14:00:18.867546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 14:00:18.867546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [parameters: {'id': 'b936d5f9932144eb96bf14d8300ad3c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 14:00:18.867546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81605) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-57e83d83-7889-4cef-9c50-9154fda800cf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b936d5f9932144eb96bf14d8300ad3c3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b936d5f9932144eb96bf14d8300ad3c3. Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 14:00:18.876162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 14:00:18.877219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application yield resource Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise self.value Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application session.commit() Sep 13 14:00:18.878495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.flush() Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application compat.raise_( Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 14:00:18.879528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 14:00:18.880430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 14:00:18.881217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [parameters: {'id': 'b936d5f9932144eb96bf14d8300ad3c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:18.882213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:18.883168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b936d5f9932144eb96bf14d8300ad3c3. Sep 13 14:00:18.884323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:18.886772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 478/949] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Sep 13 14:00:18 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:18.897120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2820a2f-8a5c-4448-ad00-2411c5e5bb40 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:18.974827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d16538-9f8e-4200-a8e4-f45577d700ab None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:18.975488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d16538-9f8e-4200-a8e4-f45577d700ab None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:18.975963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d16538-9f8e-4200-a8e4-f45577d700ab None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:18.985063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 479/950] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:18.987174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fb9bd-c4a5-4a93-9bbd-d9390bd750c2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:18.987780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fb9bd-c4a5-4a93-9bbd-d9390bd750c2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:18.988274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fb9bd-c4a5-4a93-9bbd-d9390bd750c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:18.995851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:18.998057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2820a2f-8a5c-4448-ad00-2411c5e5bb40 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.012251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-169fb9bd-c4a5-4a93-9bbd-d9390bd750c2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:19.015209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-169fb9bd-c4a5-4a93-9bbd-d9390bd750c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:19.038293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.039017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.039366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.039924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.047640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.049195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:19.062370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:19.063366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:19.064613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 472/951] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:19.069863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.082969 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:19.121005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 480/952] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:19.123125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fb9bd-c4a5-4a93-9bbd-d9390bd750c2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.187182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2df3c-f919-46b5-bd14-a2e56c8dddc8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.224564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20550e32-c7ce-43ce-b71a-d73274fc44be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.225239 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550e32-c7ce-43ce-b71a-d73274fc44be None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.225453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550e32-c7ce-43ce-b71a-d73274fc44be None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.225747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20550e32-c7ce-43ce-b71a-d73274fc44be None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.226407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550e32-c7ce-43ce-b71a-d73274fc44be None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:19.226708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550e32-c7ce-43ce-b71a-d73274fc44be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.228120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20550e32-c7ce-43ce-b71a-d73274fc44be None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.236979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fb9bd-c4a5-4a93-9bbd-d9390bd750c2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.246473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 473/953] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:19.247680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8c0a-be37-4550-9de7-c96355eac9d0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.247961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8c0a-be37-4550-9de7-c96355eac9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.248330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f8c0a-be37-4550-9de7-c96355eac9d0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.249521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 474/954] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:19.255345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fe4ff-44f7-421e-9933-ba2b098e7182 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.255622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fe4ff-44f7-421e-9933-ba2b098e7182 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.255898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fe4ff-44f7-421e-9933-ba2b098e7182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.275488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-439fe4ff-44f7-421e-9933-ba2b098e7182 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:19.276695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-439fe4ff-44f7-421e-9933-ba2b098e7182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:19.291535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.293011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:19.293594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.295033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.296242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.297259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.300428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:19.312135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:19.313365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:19.314331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 481/955] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:19.323221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.375131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 475/956] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:19.383831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:19.409165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fe4ff-44f7-421e-9933-ba2b098e7182 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.450006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a3146-bc9a-4f82-9ad8-f11b911f30ff None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.488319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.489092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.489894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.490172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.490999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:19.491306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.492799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.499755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 482/957] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:19.506443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.539120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fe4ff-44f7-421e-9933-ba2b098e7182 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.580143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.580786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.581112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.581389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.592399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:19.597351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:19.598409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:19.602588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 476/958] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:19.606691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.634756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bf937-0a14-4ca5-aa9d-f669672fe000 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.638089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:19.647992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:19.676752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.677356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.677631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.677933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.678568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:19.678849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.680233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.684916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 483/959] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:19.685708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.719474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20b614-5491-42ec-9d23-b9d1049a1983 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.757569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.758486 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.758769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.759044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.759712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:19.760028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.761387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.770068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 477/960] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:19.772587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.801577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d473d-d269-4bc1-9459-af92afc7512c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.836737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.837363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.837642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.837945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.838525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.839876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.848689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:19.849759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:19.850880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:19.851924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:19.852775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 484/961] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:19.853319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.884947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c451344-9270-4a06-af5f-102bc56f87e8 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:19.893515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:19.921365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:19.921997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:19.922277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:19.922549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:19.923211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:19.924573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:19.953774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 478/962] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:19 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:19.953774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:19.968908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0db097-63ea-4837-bb4f-4764e756fb9d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:20.019667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:20.020338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:20.020684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:20.021025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:20.034120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:20.035923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:20.053676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:20.054938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:20.056195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:20.056195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:20.056195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:20.058340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 485/963] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:20.063021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:20.090709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff423-2954-4c00-9e18-b601cbd8792f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:20.122801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:20.135122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:20.135890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:20.136246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:20.136599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:20.137317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] RBAC: Adding query filter params (name=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:20.137698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:20.139435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:20.150069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 479/964] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:20.151269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:20.189874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:20.209099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435bb53-6f08-4ebd-bfd9-83e0f2716783 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:20.245907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:20.246502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:20.246782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:20.247079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:20.247790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:20.248112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:20.249664 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:20.262423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 486/965] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:20.268409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2cea1-07eb-4ba3-bbd8-0baad836fcde None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:20.268586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:20.312817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:20.313539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:20.313955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:20.314433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:20.315237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:20.315618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:20.317319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:20.323255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 480/966] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:20.384236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f559fc-0ea1-4c72-8045-386af015c547 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:20.424664 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:20.425659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:20.426165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:20.426612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:20.435098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:20.437066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:20.450843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:20.452452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 14:00:20.453432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:20.455684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 487/967] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:00:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:20.462908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:20.526964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:20.575774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2232b0-0e45-4ade-9b73-211aa931fd4d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:20.611422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:20.612266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:20.612714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:20.613132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:20.614070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:20.614532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:20.616105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:20.634488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 481/968] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Sep 13 14:00:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:20.640388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:20.644504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:20.744527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a8056-fa29-4625-a797-e4ec99a2e8fd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:20.784561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:20.785170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:20.785447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:20.785720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:20.856121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:20.857643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:20.860591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:20.960511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 488/969] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:20 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:20.967599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:21.071086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab9945-abcb-4bc9-a8e0-4598a35b1a51 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:21.084884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:21.107367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:21.108462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:21.108774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:21.109010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:21.119736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:21.121181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:21.136994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:21.138040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:21.139167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:21.139167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:21.139167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:21.139167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 482/970] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:21.143900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:21.248919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8aa3f-a2b5-42c9-84b3-ac3f3f2ee088 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:21.251395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:21.286786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:21.287391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:21.287666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:21.287935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:21.288579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:21.288880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:21.290275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:21.304962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 489/971] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:21.310725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:21.416941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403e42e-8653-4fe8-bfb5-b2cdbf5cacc2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:21.451262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:21.452513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:21.452794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:21.453068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:21.459859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:21.461283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:21.472658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:21.474716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:21.475919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 483/972] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:21.479544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:21.598000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b985c3-16d2-4ba1-8bf7-c77cb7ab3d7c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:21.632368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fca01d-8a4b-4c49-b0c0-83535c0745ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:21.632971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fca01d-8a4b-4c49-b0c0-83535c0745ce None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:21.633245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fca01d-8a4b-4c49-b0c0-83535c0745ce None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:21.633513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fca01d-8a4b-4c49-b0c0-83535c0745ce None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:21.634223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fca01d-8a4b-4c49-b0c0-83535c0745ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:21.634523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fca01d-8a4b-4c49-b0c0-83535c0745ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:21.635931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fca01d-8a4b-4c49-b0c0-83535c0745ce None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:21.648598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 490/973] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:21.655455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:21.673819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea726d-a39c-4706-82c4-e7d1114934e3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:21.673819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea726d-a39c-4706-82c4-e7d1114934e3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:21.673819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea726d-a39c-4706-82c4-e7d1114934e3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:21.675771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 491/974] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:21.681691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548e939-70ba-4834-a5fc-bbda80a4e1ea None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:21.682182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548e939-70ba-4834-a5fc-bbda80a4e1ea None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:21.682612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548e939-70ba-4834-a5fc-bbda80a4e1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:21.684584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:21.702947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a548e939-70ba-4834-a5fc-bbda80a4e1ea None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:21.704839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a548e939-70ba-4834-a5fc-bbda80a4e1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:21.729591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:21.782990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f0e4a-0937-4e05-b21b-3caf5ae4d255 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:21.813942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 492/975] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:21.818930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:21.819538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:21.819816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:21.820086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:21.820803 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] RBAC: Adding query filter params (role.id=143ac081ca204690a023a1a2c385de76, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=929edc18c5874105a349dcc2b4cad14c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:21.821106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:21.822544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:21.828698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 484/976] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:21 2022] GET /identity/v3/role_assignments?user.id=929edc18c5874105a349dcc2b4cad14c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=143ac081ca204690a023a1a2c385de76 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:22.087781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548e939-70ba-4834-a5fc-bbda80a4e1ea None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:22.197333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548e939-70ba-4834-a5fc-bbda80a4e1ea None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:22.238950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:22.239914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.240418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.240911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.241831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:22.243477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:22.261154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.sql.core [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 14:00:22.261154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 14:00:22.261154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [parameters: {'user_id': 'd738f09c09344c9d9415e16a64f72ecf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 14:00:22.261154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81605) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-86e49545-756b-45f0-a7a5-d7735eb4b2a9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 13 14:00:22.269518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 14:00:22.270593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application yield resource Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise self.value Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application session.commit() Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 14:00:22.271839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.flush() Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application compat.raise_( Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 14:00:22.273011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 14:00:22.274158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 14:00:22.275098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd738f09c09344c9d9415e16a64f72ecf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:22.275956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 14:00:22.276788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 14:00:22.277720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 14:00:22.277720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 14:00:22.277720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 14:00:22.277720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 14:00:22.277720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 14:00:22.277720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.280486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 493/977] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Sep 13 14:00:22 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:22.286702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:22.323205 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb5a4-29ba-4b1d-a8a4-7c96b3dc1c3b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.323486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb5a4-29ba-4b1d-a8a4-7c96b3dc1c3b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.323761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531eb5a4-29ba-4b1d-a8a4-7c96b3dc1c3b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.328396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 494/978] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:22.334029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8f5a-a38e-4ebe-bb43-326ac2f1717a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.334315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8f5a-a38e-4ebe-bb43-326ac2f1717a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.334593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8f5a-a38e-4ebe-bb43-326ac2f1717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.355821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f95b8f5a-a38e-4ebe-bb43-326ac2f1717a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:22.357394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f95b8f5a-a38e-4ebe-bb43-326ac2f1717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:22.370318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:22.399077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:22.416364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be43ca2-0f11-4d9b-b2a4-aabdb6b28397 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:22.445861 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 495/979] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:22.462734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c54980-30ee-4b41-97d9-bc84fdf8aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:22.463670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c54980-30ee-4b41-97d9-bc84fdf8aa55 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.464135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c54980-30ee-4b41-97d9-bc84fdf8aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.464592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c54980-30ee-4b41-97d9-bc84fdf8aa55 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.473143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c54980-30ee-4b41-97d9-bc84fdf8aa55 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:22.475376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c54980-30ee-4b41-97d9-bc84fdf8aa55 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a4c54980-30ee-4b41-97d9-bc84fdf8aa55 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:22.487690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:22.488666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 14:00:22.489630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:22.491725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 485/980] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:00:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:22.497571 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8f5a-a38e-4ebe-bb43-326ac2f1717a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:22.612790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8f5a-a38e-4ebe-bb43-326ac2f1717a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:22.649619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:22.650501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.650988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.651413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.652285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:22.652737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:22.654817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:22.674307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b6c4-2fc3-47b7-9a26-bc4f4e21aff7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.674593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b6c4-2fc3-47b7-9a26-bc4f4e21aff7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.674873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3b6c4-2fc3-47b7-9a26-bc4f4e21aff7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.676497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 486/981] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:22.681463 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0fd94-d282-45e9-a724-06dbecf3512f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.681758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0fd94-d282-45e9-a724-06dbecf3512f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.682056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0fd94-d282-45e9-a724-06dbecf3512f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.697868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 496/982] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Sep 13 14:00:22 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:22.701108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-41a0fd94-d282-45e9-a724-06dbecf3512f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:22.702647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-41a0fd94-d282-45e9-a724-06dbecf3512f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:22.749448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:22.755494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:22.793925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 487/983] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:22.869995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59521094-a21b-4a72-9efd-6792933b49ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:22.907959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:22.908591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:22.908874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:22.909147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:22.909744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:22.911109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:22.919993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:22.922015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:22.922896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 497/984] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:22.928357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0fd94-d282-45e9-a724-06dbecf3512f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:23.016354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:23.041069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0fd94-d282-45e9-a724-06dbecf3512f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:23.072073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:23.078350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:23.078971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:23.079253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:23.079531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:23.080163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:23.080468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:23.081864 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:23.089769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 488/985] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:23.093407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:23.152219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:23.191032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c09cd-cac4-4e69-9194-4d6efd575ba8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:23.206586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774bf35-d8a7-45c3-8edd-9961f81deb59 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:23.227293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:23.227896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:23.228187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:23.228456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:23.229111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:23.230512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:23.244516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:23.245180 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:23.245465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:23.245755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:23.246457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:23.249760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:23.249897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 498/986] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:00:23 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:00:23.250079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:23.258620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 489/987] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:23.264689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:23.385642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0ccbf-db6f-4f3d-bb4a-0b8d01514efb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:23.412614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:23.428210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc879d-51ec-42f4-8847-adaebfa2d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:23.428829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc879d-51ec-42f4-8847-adaebfa2d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:23.429109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc879d-51ec-42f4-8847-adaebfa2d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:23.429386 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc879d-51ec-42f4-8847-adaebfa2d4c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:23.430146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc879d-51ec-42f4-8847-adaebfa2d4c6 None admin] RBAC: Adding query filter params (interface=admin, service_id=9aad3b2290c546ab88bfbc143b843fe9, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:23.430453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc879d-51ec-42f4-8847-adaebfa2d4c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:23.431819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc879d-51ec-42f4-8847-adaebfa2d4c6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:23.438184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 499/988] 10.222.0.20 () {56 vars in 1249 bytes} [Tue Sep 13 14:00:23 2022] GET /identity/v3/endpoints?service_id=9aad3b2290c546ab88bfbc143b843fe9&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:23.444087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:23.454377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:23.553989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe9ec7-6c95-4919-947a-4f11354b4480 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:23.589150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e001f6-3d30-4e31-a9da-569561a4ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:23.589789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e001f6-3d30-4e31-a9da-569561a4ea37 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:23.590169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e001f6-3d30-4e31-a9da-569561a4ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:23.590453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e001f6-3d30-4e31-a9da-569561a4ea37 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:23.591061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e001f6-3d30-4e31-a9da-569561a4ea37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:23.593401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e001f6-3d30-4e31-a9da-569561a4ea37 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:23.605809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 490/989] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 14:00:23 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:00:24.105375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7dcc-6e8b-4646-941d-0dad449f7c2a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:24.105932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7dcc-6e8b-4646-941d-0dad449f7c2a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:24.105932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7dcc-6e8b-4646-941d-0dad449f7c2a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:24.107419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 500/990] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:24.113159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a77ec6-eed2-4dcf-9215-6531f0e148f4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:24.113453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a77ec6-eed2-4dcf-9215-6531f0e148f4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:24.113755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a77ec6-eed2-4dcf-9215-6531f0e148f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:24.134286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-84a77ec6-eed2-4dcf-9215-6531f0e148f4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:24.135486 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-84a77ec6-eed2-4dcf-9215-6531f0e148f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:24.201986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:24.209862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 491/991] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:24.474621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca7dcc-6e8b-4646-941d-0dad449f7c2a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:24.587869 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca7dcc-6e8b-4646-941d-0dad449f7c2a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:24.621456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:24.622304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:24.622732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:24.623709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:24.624446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:24.625966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:24.634904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:24.636109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:24.637170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:24.644790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 501/992] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Sep 13 14:00:24 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:24.646372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a77ec6-eed2-4dcf-9215-6531f0e148f4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:24.743715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a77ec6-eed2-4dcf-9215-6531f0e148f4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:24.777467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9636e4-4a10-4f12-9f25-104046a11c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:24.778645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9636e4-4a10-4f12-9f25-104046a11c9b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:24.779095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9636e4-4a10-4f12-9f25-104046a11c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:24.779608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9636e4-4a10-4f12-9f25-104046a11c9b None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:24.780383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9636e4-4a10-4f12-9f25-104046a11c9b None admin] RBAC: Adding query filter params (name=image) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:24.780834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9636e4-4a10-4f12-9f25-104046a11c9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:24.782361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9636e4-4a10-4f12-9f25-104046a11c9b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:24.787455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 492/993] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:24.793135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:24.848777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:24.897940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a4978-c63d-48e1-a07d-85a8910c8e06 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:24.932116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc571dae-fd72-44cd-b521-03c7d0130d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:24.932714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571dae-fd72-44cd-b521-03c7d0130d8d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:24.932989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571dae-fd72-44cd-b521-03c7d0130d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:24.933259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc571dae-fd72-44cd-b521-03c7d0130d8d None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:24.933901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571dae-fd72-44cd-b521-03c7d0130d8d None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:24.934201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571dae-fd72-44cd-b521-03c7d0130d8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:24.935585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc571dae-fd72-44cd-b521-03c7d0130d8d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:24.940744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 502/994] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:25.441708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536956a8-b5d1-4a85-9b15-7a47a0362dd4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:25.442046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536956a8-b5d1-4a85-9b15-7a47a0362dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:25.442290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536956a8-b5d1-4a85-9b15-7a47a0362dd4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:25.443805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 493/995] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:25.448704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af1749-c32a-4385-b49a-190433ee179f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:25.449138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af1749-c32a-4385-b49a-190433ee179f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:25.449301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af1749-c32a-4385-b49a-190433ee179f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:25.467826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-75af1749-c32a-4385-b49a-190433ee179f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:25.469009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-75af1749-c32a-4385-b49a-190433ee179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:25.542384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 503/996] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:25.747209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789d49c-e5c4-4e77-9bbd-763738ec5b3d None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:25.747726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789d49c-e5c4-4e77-9bbd-763738ec5b3d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:25.747726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1789d49c-e5c4-4e77-9bbd-763738ec5b3d None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:25.749982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 494/997] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:25.754934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a142b61-11fb-417a-bb1d-a3177f387a51 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:25.755221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a142b61-11fb-417a-bb1d-a3177f387a51 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:25.755493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a142b61-11fb-417a-bb1d-a3177f387a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:25.773152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9a142b61-11fb-417a-bb1d-a3177f387a51 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:25.774348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a142b61-11fb-417a-bb1d-a3177f387a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:25.808824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789d49c-e5c4-4e77-9bbd-763738ec5b3d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:25.821581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:25.847938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 504/998] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:25.878852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:25.918098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1789d49c-e5c4-4e77-9bbd-763738ec5b3d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:25.951767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:25.952387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:25.952666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:25.952938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:25.953514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:25.954904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:25.963656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:25.964659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:25.965508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:25.967681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 495/999] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:25.971447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a142b61-11fb-417a-bb1d-a3177f387a51 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.070713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a142b61-11fb-417a-bb1d-a3177f387a51 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.104627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.105229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.105508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.105802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.106506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:26.106807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.108683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.113578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 505/1000] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:26.118959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.127045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.171177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:26.229108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948320d8-6b54-4e5e-bf93-6998c31e4889 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.229664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da3999-0098-43d9-84aa-f312b020864a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.256428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:26.264312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.264934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.265205 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.265518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.266111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.266679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.267279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.267442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.267595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.267828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.268447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:26.268750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.270625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.275758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 496/1001] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:26.276196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:26.277319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:26.279093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 506/1002] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Sep 13 14:00:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:26.282029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.283247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.387256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061a8e5-c116-4e16-acd7-75e7804fbc47 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.390456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63cf4-f995-4fa6-8208-da3af0759138 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.393766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:26.420905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.421510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.421808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.422083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.422783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] RBAC: Adding query filter params (interface=public, service_id=95a3157713d54112ba96953090575835, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:26.423091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.424440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.429349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 497/1003] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 14:00:26 2022] GET /identity/v3/endpoints?service_id=95a3157713d54112ba96953090575835&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:26.430532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.431337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.431868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.432338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.433148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:26.433633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.434949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.435229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.440923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 507/1004] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:26.446589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.495688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:26.546602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74f82-3d2a-4d2f-a3c2-4d60aa1828a6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.558414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c751d-5e3f-45fd-a3a8-5a455c89a1d7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.581850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.582451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.582730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.583002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.583606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.585028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.594703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8150d21-097e-42d8-a09b-2800fdc9bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.595324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8150d21-097e-42d8-a09b-2800fdc9bf5b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.595603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8150d21-097e-42d8-a09b-2800fdc9bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.595875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8150d21-097e-42d8-a09b-2800fdc9bf5b None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.596506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8150d21-097e-42d8-a09b-2800fdc9bf5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:26.596806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8150d21-097e-42d8-a09b-2800fdc9bf5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.597904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 498/1005] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 14:00:26 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:00:26.598234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8150d21-097e-42d8-a09b-2800fdc9bf5b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.603930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 508/1006] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Sep 13 14:00:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:26.613868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:26.723771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb7d6b-6644-4781-a785-4c1b046a2106 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:26.745888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:26.762570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e21e07-9a1f-48fb-8564-922b71e4a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:26.763246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e21e07-9a1f-48fb-8564-922b71e4a3fe None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:26.763512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e21e07-9a1f-48fb-8564-922b71e4a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:26.764045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e21e07-9a1f-48fb-8564-922b71e4a3fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:26.764695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e21e07-9a1f-48fb-8564-922b71e4a3fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:26.766106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e21e07-9a1f-48fb-8564-922b71e4a3fe None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:26.783517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 499/1007] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:27.101654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eca02-9b88-40fe-9870-241c10732f25 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:27.101982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eca02-9b88-40fe-9870-241c10732f25 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:27.102158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eca02-9b88-40fe-9870-241c10732f25 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:27.103558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 509/1008] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:27.109429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c470d1-d62e-4ce2-8a50-33b140f7a960 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:27.109714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c470d1-d62e-4ce2-8a50-33b140f7a960 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:27.110016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c470d1-d62e-4ce2-8a50-33b140f7a960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:27.128877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-30c470d1-d62e-4ce2-8a50-33b140f7a960 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:27.130261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-30c470d1-d62e-4ce2-8a50-33b140f7a960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:27.205032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 500/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:27 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:27.473172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eca02-9b88-40fe-9870-241c10732f25 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:27.507137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:27.588071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eca02-9b88-40fe-9870-241c10732f25 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:27.622296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff92ea7-e7df-4613-9310-2ca2d9bf32b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:27.622891 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff92ea7-e7df-4613-9310-2ca2d9bf32b7 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:27.623422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff92ea7-e7df-4613-9310-2ca2d9bf32b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:27.623691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff92ea7-e7df-4613-9310-2ca2d9bf32b7 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:27.624393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff92ea7-e7df-4613-9310-2ca2d9bf32b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:27.626839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff92ea7-e7df-4613-9310-2ca2d9bf32b7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:27.641334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 510/1010] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:00:27 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:28.790111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df83ab-485c-4eda-b4b5-033dbadb6e42 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:28.790619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df83ab-485c-4eda-b4b5-033dbadb6e42 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:28.790619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df83ab-485c-4eda-b4b5-033dbadb6e42 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:28.791888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 501/1011] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:28.798024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c0799-f1e2-4d88-800c-3580293c2de2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:28.798439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c0799-f1e2-4d88-800c-3580293c2de2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:28.798620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c0799-f1e2-4d88-800c-3580293c2de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:28.817188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-339c0799-f1e2-4d88-800c-3580293c2de2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:28.818410 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-339c0799-f1e2-4d88-800c-3580293c2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:28.896989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 511/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:28 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:28.946347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bbc81-6e6d-4908-967b-dd32e4ca10b3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:28.946628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bbc81-6e6d-4908-967b-dd32e4ca10b3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:28.947077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bbc81-6e6d-4908-967b-dd32e4ca10b3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:28.948395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 502/1013] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:28.954208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f0493-c346-46bd-81d2-b896e9faf633 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:28.954496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f0493-c346-46bd-81d2-b896e9faf633 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:28.954768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f0493-c346-46bd-81d2-b896e9faf633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:28.973676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c26f0493-c346-46bd-81d2-b896e9faf633 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:28.974882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c26f0493-c346-46bd-81d2-b896e9faf633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:28.989073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:29.046339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 512/1014] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:28 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:29.164332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bbc81-6e6d-4908-967b-dd32e4ca10b3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:29.268442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bbc81-6e6d-4908-967b-dd32e4ca10b3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:29.275880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:29.304624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:29.305174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.305455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.305748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.306348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:29.307701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:29.315768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f0493-c346-46bd-81d2-b896e9faf633 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:29.316882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:29.318184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 14:00:29.319067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:29.322038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 503/1015] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:29.323677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:29.425769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f0493-c346-46bd-81d2-b896e9faf633 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:29.432621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac69872-712d-421f-b977-33233bc6e6b0 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:29.461548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:29.462213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.462510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.462763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.463866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:29.465293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:29.467146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 513/1016] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:29.468034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:29.468640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.468916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.469186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.469846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:29.470149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:29.471695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:29.472443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:29.481065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 504/1017] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:29.483601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:29.491807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:29.521940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:29.584981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc1cb1-1538-40dd-a705-4f47bb31cec4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:29.599822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d514d-0dc1-406f-8009-f30a71626a1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:29.619902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc5b22a-d383-4974-998a-3078d5f73d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:29.620883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b22a-d383-4974-998a-3078d5f73d6d None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.621380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b22a-d383-4974-998a-3078d5f73d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.621722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc5b22a-d383-4974-998a-3078d5f73d6d None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.622392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5b22a-d383-4974-998a-3078d5f73d6d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:29.624335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc5b22a-d383-4974-998a-3078d5f73d6d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:29.635809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:29.636414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.636692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.636965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.637590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:29.637925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:29.639410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:29.644558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 505/1018] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Sep 13 14:00:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:29.651132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:29.659090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 514/1019] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:00:29 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:00:29.761663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:29.767003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972fe09-a2ab-43f0-8f36-41c8a0627217 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:29.804152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae207d-c4df-420a-bc01-a44a64b42ed4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.804706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae207d-c4df-420a-bc01-a44a64b42ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.804706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae207d-c4df-420a-bc01-a44a64b42ed4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.805757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:29.806325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 515/1020] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:29.806435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.806639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.806908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.807575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:29.808957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:29.815591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2146-efbf-42c7-94e6-74161fa242ca None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:29.815914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2146-efbf-42c7-94e6-74161fa242ca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:29.816144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2146-efbf-42c7-94e6-74161fa242ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:29.834608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 506/1021] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:29.835995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-56be2146-efbf-42c7-94e6-74161fa242ca None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:29.837128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-56be2146-efbf-42c7-94e6-74161fa242ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:29.913442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:29.925632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 516/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:30.197324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:30.297865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df2832-8c7f-44fe-a7aa-4a8bbe9e86ca None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:30.331265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:30.331888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:30.332162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:30.332428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] PATH_INFO: `/v3/services/image` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:30.333115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:30.334377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:30.343058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:30.344091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:30.344966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 507/1023] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Sep 13 14:00:30 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:30.350335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be2146-efbf-42c7-94e6-74161fa242ca None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:30.460577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be2146-efbf-42c7-94e6-74161fa242ca None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:30.495714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:30.496322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:30.496605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:30.496878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:30.497507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] RBAC: Adding query filter params (name=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:30.497908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:30.499376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:30.509075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 517/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:30.510062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:30.531086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:30.616113 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe05ee-c2a8-4f3d-94de-f7128d533797 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:30.652410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:30.653022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:30.653303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:30.653861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:30.655027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] RBAC: Adding query filter params (type=image) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:30.655331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:30.656693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:30.666523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 508/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:30.667763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:30.734458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:30.777786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01bc50-4130-4390-9f59-7d2f16118c14 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:30.814222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102ca9e-1580-4fed-923b-a02413db7a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:30.814868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102ca9e-1580-4fed-923b-a02413db7a36 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:30.815108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102ca9e-1580-4fed-923b-a02413db7a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:30.815384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102ca9e-1580-4fed-923b-a02413db7a36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:30.816110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102ca9e-1580-4fed-923b-a02413db7a36 None admin] RBAC: Adding query filter params (interface=public, service_id=71375708a48947e5afde39616b5820a0, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:30.816421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102ca9e-1580-4fed-923b-a02413db7a36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:30.817858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102ca9e-1580-4fed-923b-a02413db7a36 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:30.829188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 518/1026] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 14:00:30 2022] GET /identity/v3/endpoints?service_id=71375708a48947e5afde39616b5820a0&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:30.830185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:30.911910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:30.930974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5032f-4a1f-4d8f-ab3f-b7aaba8ef1cd None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:30.965406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90459960-2135-4693-926b-5eea102db368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:30.966103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90459960-2135-4693-926b-5eea102db368 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:30.966386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90459960-2135-4693-926b-5eea102db368 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:30.966656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90459960-2135-4693-926b-5eea102db368 None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:30.967988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90459960-2135-4693-926b-5eea102db368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:30.969358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90459960-2135-4693-926b-5eea102db368 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:30.981984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 509/1027] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 14:00:30 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:00:31.905597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7b95-31e1-4e16-8ed1-94ea3f805a72 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:31.906153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7b95-31e1-4e16-8ed1-94ea3f805a72 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:31.906153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e7b95-31e1-4e16-8ed1-94ea3f805a72 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:31.907605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 519/1028] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:31.912715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966d21-9e91-45b1-8d21-6689392fec37 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:31.913095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966d21-9e91-45b1-8d21-6689392fec37 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:31.913272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966d21-9e91-45b1-8d21-6689392fec37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:31.932751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-84966d21-9e91-45b1-8d21-6689392fec37 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:31.933956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-84966d21-9e91-45b1-8d21-6689392fec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:32.007502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 510/1029] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:32.104338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc68c03-cfa2-4f8e-9482-3ffd9c788386 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:32.104679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc68c03-cfa2-4f8e-9482-3ffd9c788386 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:32.104889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc68c03-cfa2-4f8e-9482-3ffd9c788386 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:32.106409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 520/1030] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:32.111986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4164bd-8614-4670-8f9e-f4c0b0f35d4e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:32.112283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4164bd-8614-4670-8f9e-f4c0b0f35d4e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:32.112562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4164bd-8614-4670-8f9e-f4c0b0f35d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:32.130501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cd4164bd-8614-4670-8f9e-f4c0b0f35d4e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:32.131675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cd4164bd-8614-4670-8f9e-f4c0b0f35d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:32.206048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 511/1031] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:32.283017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc68c03-cfa2-4f8e-9482-3ffd9c788386 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:32.385212 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc68c03-cfa2-4f8e-9482-3ffd9c788386 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:32.408659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:32.426641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:32.427269 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:32.427543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:32.427807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:32.428903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:32.430566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:32.432289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 521/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:32.438811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4164bd-8614-4670-8f9e-f4c0b0f35d4e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:32.447541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:32.484685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:32.554348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4164bd-8614-4670-8f9e-f4c0b0f35d4e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:32.589465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe61ad-badd-4143-8857-28ebce46930b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:32.591410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:32.593135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:32.593135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:32.593135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:32.593792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:32.595266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:32.619634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 512/1033] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 14:00:32 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:32.628874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:32.629527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:32.629828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:32.630097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:32.643402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:32.648405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:32.649452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:32.650815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 522/1034] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 13 14:00:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:32.656194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:32.729875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:32.774646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2f07f-e261-40a0-9f89-0bcef4ccaef2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:32.812706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:32.813313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:32.813589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:32.813902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:32.814557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:32.814859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:32.816230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:32.824926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 513/1035] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 13 14:00:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:32.830611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:32.884796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:32.937130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff5cda-175c-4134-888f-ed3867e965ab None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:32.972459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da2c56-fe81-47bc-94e1-26a82f972d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:32.973099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2c56-fe81-47bc-94e1-26a82f972d2e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:32.973434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2c56-fe81-47bc-94e1-26a82f972d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:32.973711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da2c56-fe81-47bc-94e1-26a82f972d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:32.984623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da2c56-fe81-47bc-94e1-26a82f972d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:32.986094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da2c56-fe81-47bc-94e1-26a82f972d2e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-87da2c56-fe81-47bc-94e1-26a82f972d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:33.000991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:33.002256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:33.003158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:33.003158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:33.003158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:33.003158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 523/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:33.009492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.113452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10edf5a-a6b5-4d90-a802-1927a4b7ad65 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:33.133875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42ffae-92b6-4ab5-8d3b-19861b1b3b75 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.134166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42ffae-92b6-4ab5-8d3b-19861b1b3b75 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.134422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42ffae-92b6-4ab5-8d3b-19861b1b3b75 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.136012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 524/1037] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:33.137257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:33.141625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56a8f0-f40a-4df0-a067-a8a6d6b0ce31 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.141948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56a8f0-f40a-4df0-a067-a8a6d6b0ce31 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.142228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56a8f0-f40a-4df0-a067-a8a6d6b0ce31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.149859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:33.150468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.150748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.151018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.151661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:33.151964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:33.153306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:33.161275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9d56a8f0-f40a-4df0-a067-a8a6d6b0ce31 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:33.162500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9d56a8f0-f40a-4df0-a067-a8a6d6b0ce31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:33.167191 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 514/1038] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:33.172257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.236966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 525/1039] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:33.278356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba8442-79b8-43cc-a433-5acff2fccbde None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:33.312882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:33.313509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.313815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.314089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.321082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:33.322540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:33.334934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:33.336007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:33.337646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 515/1040] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:33.341914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56a8f0-f40a-4df0-a067-a8a6d6b0ce31 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.373541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:33.444642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56a8f0-f40a-4df0-a067-a8a6d6b0ce31 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:33.479313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:33.479922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.480198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.480465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.481187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:33.481488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:33.483123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:33.505417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 526/1041] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:33.507188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.508789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.560006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:33.614760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:33.626549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0c9d1-cc81-42b2-ae4a-4b08f039b0f2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:33.627146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709514e8-e2dc-4f41-8d54-7a5b59c858cd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:33.661672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:33.662302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.662579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.662862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.668747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:33.669380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.669653 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.670700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.671600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:33.673402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:33.683013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:33.684013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:33.687162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 527/1042] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:33.690199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.739047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:33.740502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:33.796542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fb24b-9590-477d-b059-b8aefcc41243 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:33.798075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:33.831970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:33.832573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:33.832851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:33.833120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:33.833755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] RBAC: Adding query filter params (name=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:33.834067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:33.835412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:33.841086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 528/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:33.846312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.869590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 516/1044] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:33.870864 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:33.871022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:33.962219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82b011-554d-42d3-bcaa-3e219b55755d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:33.984065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbe324-df16-4fa8-9c9b-0dccbf061dd5 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:34.001802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:34.001802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.001802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.001802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.002213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:34.002213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:34.002521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:34.010440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 529/1045] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:34.018785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:34.023712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:34.024340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.024616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.024885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.035797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:34.037222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:34.040006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:34.053275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:34.054319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:34.055221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:34.055221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:34.055221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:34.055221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 517/1046] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:34.061315 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:34.110146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:34.134120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1e16-0b3e-43c1-b914-9a93cc8310ba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:34.171688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:34.172307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.172579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.172854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.173521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:34.174929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:34.176085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6736a4-fdde-4a22-90d5-b965894eff39 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:34.195342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 530/1047] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:00:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:00:34.222460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:34.222998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.223274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.223541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.224218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:34.224519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:34.225935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:34.240566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 518/1048] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:34.246118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:34.368458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cb953-3d31-41ce-9b0c-f447c347467c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:34.406545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:34.407182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.407461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.407731 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.415359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:34.418012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:34.421648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:34.433827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:34.434943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:34.436174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 531/1049] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:34.441913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:34.497475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:34.552173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409146-5b90-4c20-b8ee-66e4fca75dd9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:34.587631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c40d5b-c4bf-458d-bf03-fca7139ed1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:34.588243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40d5b-c4bf-458d-bf03-fca7139ed1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.588524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40d5b-c4bf-458d-bf03-fca7139ed1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.588798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40d5b-c4bf-458d-bf03-fca7139ed1b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.589489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c40d5b-c4bf-458d-bf03-fca7139ed1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:34.589823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c40d5b-c4bf-458d-bf03-fca7139ed1b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:34.591248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c40d5b-c4bf-458d-bf03-fca7139ed1b7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:34.604038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 519/1050] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:34.610388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:34.712689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253ad04-c09d-4c50-9b13-a4e20ddfb297 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:34.747306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230131cd-5d92-4a9a-a332-69a9b00350de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:34.748058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230131cd-5d92-4a9a-a332-69a9b00350de None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.748316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230131cd-5d92-4a9a-a332-69a9b00350de None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.748643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230131cd-5d92-4a9a-a332-69a9b00350de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.749406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230131cd-5d92-4a9a-a332-69a9b00350de None admin] RBAC: Adding query filter params (role.id=143ac081ca204690a023a1a2c385de76, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=6a4ea803f6e44648bb217abd00e11c3c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:34.749721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230131cd-5d92-4a9a-a332-69a9b00350de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:34.751242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230131cd-5d92-4a9a-a332-69a9b00350de None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:34.765756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 532/1051] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:34 2022] GET /identity/v3/role_assignments?user.id=6a4ea803f6e44648bb217abd00e11c3c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=143ac081ca204690a023a1a2c385de76 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:34.876285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b76d6-da7d-4e4c-9ab9-481e709942b7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.876618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b76d6-da7d-4e4c-9ab9-481e709942b7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.876845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b76d6-da7d-4e4c-9ab9-481e709942b7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.878405 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 520/1052] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:34.884414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f12552-8d2a-4788-843e-220f686c190e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:34.884735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f12552-8d2a-4788-843e-220f686c190e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:34.884985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f12552-8d2a-4788-843e-220f686c190e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:34.905554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d4f12552-8d2a-4788-843e-220f686c190e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:34.906811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d4f12552-8d2a-4788-843e-220f686c190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:34.969983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:34.981618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 533/1053] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:35.255241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b76d6-da7d-4e4c-9ab9-481e709942b7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:35.355840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b76d6-da7d-4e4c-9ab9-481e709942b7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:35.370064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:35.392569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:35.393337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:35.393612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:35.393910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:35.404957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:35.409696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:35.410714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:35.411563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:35.413809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 521/1054] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:35.417031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f12552-8d2a-4788-843e-220f686c190e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:35.516666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f12552-8d2a-4788-843e-220f686c190e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:35.556311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:35.556915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:35.557190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:35.557456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:35.558132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:35.558431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:35.559775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:35.568781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 534/1055] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:35.574619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:35.678674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac8187-e2a3-4494-b0aa-d2bdbf5450ef None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:35.711706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:35.712320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:35.712589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:35.712856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] PATH_INFO: `/v3/users/de239ca2780d41a29fa2e8c7a5df0514` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:35.725664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=de239ca2780d41a29fa2e8c7a5df0514)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:35.727145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:35.728910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 522/1056] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:35 2022] GET /identity/v3/users/de239ca2780d41a29fa2e8c7a5df0514 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:35.734538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:35.790403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:35.836496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2b581-3d01-4e16-9972-8c810f0edf9b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:35.868963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:35.869562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:35.869868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:35.870138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:35.938594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:35.940096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:35.993060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:36.040816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 535/1057] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:36.047204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:36.077442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:36.147490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb56f-a2b2-44ad-a921-a040d44f6ca9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:36.181134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7506e6-d085-41ca-a5de-42776f12f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:36.181779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7506e6-d085-41ca-a5de-42776f12f35e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:36.182060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7506e6-d085-41ca-a5de-42776f12f35e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:36.182330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7506e6-d085-41ca-a5de-42776f12f35e None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:36.183425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7506e6-d085-41ca-a5de-42776f12f35e None admin] RBAC: Authorizing `identity:get_project(project_id=acb9381780194f6aa6e4f661df696687)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:36.184853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7506e6-d085-41ca-a5de-42776f12f35e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:36.187328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 523/1058] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:00:36 2022] GET /identity/v3/projects/acb9381780194f6aa6e4f661df696687 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:36.193121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:36.291973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd3da0-1433-44c9-a5a8-c1818c98c49e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:36.331035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535bbdd-9d3a-420d-9ce1-a46f9e8c10d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:36.331638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535bbdd-9d3a-420d-9ce1-a46f9e8c10d2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:36.331913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535bbdd-9d3a-420d-9ce1-a46f9e8c10d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:36.332177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535bbdd-9d3a-420d-9ce1-a46f9e8c10d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:36.332892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535bbdd-9d3a-420d-9ce1-a46f9e8c10d2 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=acb9381780194f6aa6e4f661df696687, user.id=de239ca2780d41a29fa2e8c7a5df0514) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:36.333194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535bbdd-9d3a-420d-9ce1-a46f9e8c10d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:36.334625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535bbdd-9d3a-420d-9ce1-a46f9e8c10d2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:36.345813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 536/1059] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:36 2022] GET /identity/v3/role_assignments?user.id=de239ca2780d41a29fa2e8c7a5df0514&scope.project.id=acb9381780194f6aa6e4f661df696687&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:36.416525 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515132-93e7-4f25-849d-5faccc1d26bb None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:36.417073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515132-93e7-4f25-849d-5faccc1d26bb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:36.417073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515132-93e7-4f25-849d-5faccc1d26bb None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:36.418636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 524/1060] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:36.424752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beee521-c97d-4987-96e3-274af288c5ee None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:36.425033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beee521-c97d-4987-96e3-274af288c5ee None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:36.425309 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beee521-c97d-4987-96e3-274af288c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:36.444371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6beee521-c97d-4987-96e3-274af288c5ee None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:36.445584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6beee521-c97d-4987-96e3-274af288c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:36.460513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:36.523735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 537/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:36.792892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515132-93e7-4f25-849d-5faccc1d26bb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:36.898070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:36.904787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515132-93e7-4f25-849d-5faccc1d26bb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:36.918907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa674-9dc9-4ff9-9dc7-a877c2b7b286 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:36.919185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa674-9dc9-4ff9-9dc7-a877c2b7b286 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:36.919462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa674-9dc9-4ff9-9dc7-a877c2b7b286 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:36.921646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 538/1062] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:36.926414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35658-d64f-4ad4-be1a-5c7c55fbb0a2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:36.926700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35658-d64f-4ad4-be1a-5c7c55fbb0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:36.926963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d35658-d64f-4ad4-be1a-5c7c55fbb0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:36.942936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:36.943540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:36.943822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:36.944109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:36.945164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-49d35658-d64f-4ad4-be1a-5c7c55fbb0a2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:36.946375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-49d35658-d64f-4ad4-be1a-5c7c55fbb0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:37.016200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:37.017771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:37.018055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 539/1063] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:37.115107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:37.121080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 525/1064] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:37.126981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d35658-d64f-4ad4-be1a-5c7c55fbb0a2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:37.203478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:37.228198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d35658-d64f-4ad4-be1a-5c7c55fbb0a2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:37.263105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:37.263933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:37.264217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:37.264492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:37.274355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:37.275790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:37.289948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:37.290992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:37.291943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:37.292841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:37.292841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:37.292841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:37.292841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 540/1065] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:37.299517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:37.396614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975557f-9f6a-42c8-869c-46c01f571c39 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:37.397578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4d5e-e070-4b3d-b565-1bf155315594 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:37.426503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:37.433163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:37.433792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:37.434093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:37.434344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:37.434672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:37.435123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:37.435812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:37.435917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:37.436156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:37.436425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:37.437218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:37.446693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:37.449702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 541/1066] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:37.451532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:37.452475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:37.453335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 526/1067] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 13 14:00:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:37.459907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:37.580599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0a928-0459-4ce1-8987-6763a905aa87 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:37.618382 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:37.619006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:37.619286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:37.619560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:37.620233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:37.620555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:37.621939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:37.632999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 542/1068] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 13 14:00:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:37.637849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:37.705797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:37.750008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba11d-1455-4ebd-a3d6-901801f7603c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:37.784102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:37.784726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:37.784989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:37.785259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:37.795828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:37.797252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:37.814633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:37.815624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:37.816968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:37.816968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:37.816968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:37.816968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 527/1069] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:37.821391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:37.872964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:37.922969 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87682ba7-37da-4d0b-9f91-94c40ab48d16 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:37.955659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:37.956310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:37.956542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:37.956815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:37.957452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:37.957788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:37.959153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:37.971069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 543/1070] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:37.976686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.077074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a16581b-5579-43c6-84e5-41b203235e4a None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:38.087248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:38.111502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:38.112130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.112408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.112680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.119990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:38.121413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:38.132373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:38.133458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:38.134466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 528/1071] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 14:00:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:38.139646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.236530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91447d30-be4a-4304-87eb-8bfcb5ee5c25 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:38.249962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:38.271115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:38.271714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.271993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.272265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.272978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:38.273279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:38.274720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:38.291375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 544/1072] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 14:00:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:38.297685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.398053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f0d35-0c42-444c-9edc-022ba4488020 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:38.433707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876b06-607d-4855-b130-e07ffe7c9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:38.434371 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876b06-607d-4855-b130-e07ffe7c9f72 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.434630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876b06-607d-4855-b130-e07ffe7c9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.434899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876b06-607d-4855-b130-e07ffe7c9f72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.445703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876b06-607d-4855-b130-e07ffe7c9f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:38.447154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876b06-607d-4855-b130-e07ffe7c9f72 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-98876b06-607d-4855-b130-e07ffe7c9f72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:38.460988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:38.462068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:38.462978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:38.462978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:38.462978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:38.462978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 529/1073] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:38.468167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.503021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f89f13-fadf-42f4-9edd-aa23e8d66501 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.503304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f89f13-fadf-42f4-9edd-aa23e8d66501 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.503499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f89f13-fadf-42f4-9edd-aa23e8d66501 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.504824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 530/1074] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:38.509741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f10a86-b901-43be-a63b-dd411e497b67 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.510039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f10a86-b901-43be-a63b-dd411e497b67 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.510313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f10a86-b901-43be-a63b-dd411e497b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.532194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-44f10a86-b901-43be-a63b-dd411e497b67 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:38.533414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-44f10a86-b901-43be-a63b-dd411e497b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:38.549900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:38.573199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16a6d8-50e2-4d36-aa52-5a85dd70ae71 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:38.601312 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 531/1075] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:38.607990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:38.608590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.608865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.609135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.609799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:38.610100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:38.611450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:38.624137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 545/1076] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:38.629586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f10a86-b901-43be-a63b-dd411e497b67 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.726480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f10a86-b901-43be-a63b-dd411e497b67 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:38.759372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:38.759923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.760258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.760532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.766964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:38.768384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:38.779942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:38.781013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:38.782185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:38.785612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 532/1077] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:38.786795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.817323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:38.872534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.885468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d529d4-c722-4074-845b-c6880b61436a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:38.887774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:38.920441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:38.921040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:38.921321 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:38.921590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:38.922302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:38.922610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:38.924036 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:38.936974 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 546/1078] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:38.942716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:38.979474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3db753-4064-472a-b55b-0627ed04ee9f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:39.013185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:39.013831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:39.014290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:39.014565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:39.026464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:39.031049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:39.031596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:39.032700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:39.033798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 533/1079] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:00:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:39.038748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:39.044506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e50af-71d1-4be9-ad93-5836b399a3f5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:39.078518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:39.079188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:39.079464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:39.079736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/6a4ea803f6e44648bb217abd00e11c3c/roles/143ac081ca204690a023a1a2c385de76` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:39.081877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=6a4ea803f6e44648bb217abd00e11c3c, role_id=143ac081ca204690a023a1a2c385de76)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:39.083321 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:39.096134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 547/1080] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:00:38 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/6a4ea803f6e44648bb217abd00e11c3c/roles/143ac081ca204690a023a1a2c385de76 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:00:39.110516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:39.144153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857cedb-05ee-4902-a15c-d8a641be57b4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:39.182838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:39.183463 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:39.183746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:39.184021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:39.184696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:39.185001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:39.187553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:39.197486 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 534/1081] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:39.203863 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:39.309567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb57013-c78a-4e86-b4cb-e0c6b7267cf8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:39.347765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:39.348389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:39.348665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:39.348934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] PATH_INFO: `/v3/users/de239ca2780d41a29fa2e8c7a5df0514` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:39.350078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=de239ca2780d41a29fa2e8c7a5df0514)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:39.351498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:39.353068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 548/1082] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:39 2022] GET /identity/v3/users/de239ca2780d41a29fa2e8c7a5df0514 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:39.358838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:39.456593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9510bc-704a-4e5c-b289-a76d5f645d42 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:39.490767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290c2d8-001b-42db-b163-e03b871edf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:39.491388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290c2d8-001b-42db-b163-e03b871edf91 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:39.491663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290c2d8-001b-42db-b163-e03b871edf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:39.491933 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290c2d8-001b-42db-b163-e03b871edf91 None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:39.493030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290c2d8-001b-42db-b163-e03b871edf91 None admin] RBAC: Authorizing `identity:get_project(project_id=acb9381780194f6aa6e4f661df696687)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:39.494476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290c2d8-001b-42db-b163-e03b871edf91 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:39.496283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 535/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:00:39 2022] GET /identity/v3/projects/acb9381780194f6aa6e4f661df696687 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:39.502662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:39.522423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:39.605206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a158ae5-c367-4ec8-bf63-9f13ceb62eb1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:39.639068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e4a13-287f-4f41-a2ed-eb22609ea02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:39.639817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4a13-287f-4f41-a2ed-eb22609ea02e None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:39.640100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4a13-287f-4f41-a2ed-eb22609ea02e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:39.640391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4a13-287f-4f41-a2ed-eb22609ea02e None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687/users/de239ca2780d41a29fa2e8c7a5df0514/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:39.642300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e4a13-287f-4f41-a2ed-eb22609ea02e None admin] RBAC: Authorizing `identity:create_grant(project_id=acb9381780194f6aa6e4f661df696687, user_id=de239ca2780d41a29fa2e8c7a5df0514, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:39.643728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e4a13-287f-4f41-a2ed-eb22609ea02e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:39.657056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 549/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:00:39 2022] PUT /identity/v3/projects/acb9381780194f6aa6e4f661df696687/users/de239ca2780d41a29fa2e8c7a5df0514/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:00:41.229658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81955-8d84-4ef9-b560-7049d8f87420 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:41.230238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81955-8d84-4ef9-b560-7049d8f87420 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:41.230238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81955-8d84-4ef9-b560-7049d8f87420 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:41.231545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 536/1085] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:41.236846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcec14-f0ec-4b68-86fb-ec9da33d9889 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:41.237285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcec14-f0ec-4b68-86fb-ec9da33d9889 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:41.237379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcec14-f0ec-4b68-86fb-ec9da33d9889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:41.255963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fadcec14-f0ec-4b68-86fb-ec9da33d9889 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:41.257124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fadcec14-f0ec-4b68-86fb-ec9da33d9889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:41.322460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 550/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:41.591441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81955-8d84-4ef9-b560-7049d8f87420 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:41.594684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:41.692674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81955-8d84-4ef9-b560-7049d8f87420 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:41.728117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76339708-ac0a-41c2-b1b6-a7dc69f4205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:41.728766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339708-ac0a-41c2-b1b6-a7dc69f4205a None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:41.729043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339708-ac0a-41c2-b1b6-a7dc69f4205a None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:41.729310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339708-ac0a-41c2-b1b6-a7dc69f4205a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:41.740910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76339708-ac0a-41c2-b1b6-a7dc69f4205a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-76339708-ac0a-41c2-b1b6-a7dc69f4205a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:41.745820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:41.746989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:41.748554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 537/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Sep 13 14:00:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:00:41.753411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcec14-f0ec-4b68-86fb-ec9da33d9889 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:41.764168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:41.833987 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277f0e-43f4-4b17-8623-a93564c32ee2 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:41.834345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277f0e-43f4-4b17-8623-a93564c32ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:41.834628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277f0e-43f4-4b17-8623-a93564c32ee2 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:41.836524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 538/1088] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:00:41.841791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd83236-8def-46f2-a5c5-48768307cf9e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:41.842145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd83236-8def-46f2-a5c5-48768307cf9e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:41.842417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd83236-8def-46f2-a5c5-48768307cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:41.854668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcec14-f0ec-4b68-86fb-ec9da33d9889 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:41.859837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8cd83236-8def-46f2-a5c5-48768307cf9e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:41.861016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8cd83236-8def-46f2-a5c5-48768307cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:41.890611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:41.891457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:41.891736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:41.892004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:41.892658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:41.892957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:41.894513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:41.902193 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 551/1089] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 13 14:00:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:41.908359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:41.917910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:41.928949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 539/1090] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:00:41.980104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.011476 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367706b-caa4-4d26-a22f-2969a8c77933 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.043835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.044451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.045651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.045844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.056032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.057453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:42.072153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:42.073267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:42.074252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:42.074252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:42.074252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:42.074252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 552/1091] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:42.079187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd83236-8def-46f2-a5c5-48768307cf9e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.176467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd83236-8def-46f2-a5c5-48768307cf9e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.200526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.209176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.209816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.210103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.210373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.211016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:42.211319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.212664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.224677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 540/1092] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:42.230157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.282624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.301484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aeb5bf-7edc-4b9d-8e92-fad3ecf7ed4d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.323645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.330858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8ec26-c99a-4395-8487-bd008797cb23 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.335568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.336185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.336459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.336728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.347471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:00:42.353310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:42.354351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:00:42.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 553/1093] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:42.361751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.368052 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.368682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.368962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.369233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.376956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.378420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:00:42.389611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:42.390650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:42.391895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 541/1094] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:42.396350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.464920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd320b-6afb-43ff-a3ea-7ebb04c556ed None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.495922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875df0f-46d5-4be8-b643-3b7e51acb97e None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.499017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.499683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.499977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.500253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.500912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:42.501217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.502613 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.508608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.511728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 554/1095] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:42.517577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.531874 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.532480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.532766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.533039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.533791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:42.534103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.535523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.553795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 542/1096] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:42.559421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.567681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.621925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db62d1-6d0f-4e94-9984-6adab99aa431 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.655681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.655929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b7a4c-260c-4141-8aef-cb22530318af None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.656299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.656572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.656846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] PATH_INFO: `/v3/users/de239ca2780d41a29fa2e8c7a5df0514` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.658013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] RBAC: Authorizing `identity:get_user(user_id=de239ca2780d41a29fa2e8c7a5df0514)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.659419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.660990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 555/1097] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/users/de239ca2780d41a29fa2e8c7a5df0514 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:42.666701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.689245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.689898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.690178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.690442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.728687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.758288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.759735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.766848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6d166-72c8-4a00-a55a-308cdc3ab4ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.780465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.801945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.802542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.802820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.803093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:42.858460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 543/1098] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 14:00:42.864102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.872277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:42.874391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:42.930375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.961695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257cd7b-5a24-4ce6-ab2e-cf1c52bf322d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:42.973292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 556/1099] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 14:00:42.979271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:42.993235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:42.995270 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:42.995978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:42.996259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:42.996532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:43.007217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:43.008641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:00:43.023154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:43.024158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:00:43.025278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:00:43.025278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:00:43.025278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:43.025278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 544/1100] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:43.030441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:43.079385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae5736-a1b4-420b-8246-f5d4048b6da9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:43.112576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:43.113202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:43.113482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:43.113791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:43.114905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] RBAC: Authorizing `identity:get_project(project_id=acb9381780194f6aa6e4f661df696687)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:43.116336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:43.118002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 557/1101] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:00:42 2022] GET /identity/v3/projects/acb9381780194f6aa6e4f661df696687 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:43.124507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:43.128743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d67489-787a-4d17-ac1a-d5a73c3c6345 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:43.151002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:43.163894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:43.164496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:43.164778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:43.165053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:43.165694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:43.166039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:43.167398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:43.179655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 545/1102] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:43.185377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:43.203197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:43.222132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ea81-f380-46e1-81ae-f40e1b1a4b61 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:43.255784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:43.256311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:43.256591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:43.256865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:43.257581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=acb9381780194f6aa6e4f661df696687, user.id=de239ca2780d41a29fa2e8c7a5df0514) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:43.257925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:43.259359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:43.265012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 558/1103] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:43 2022] GET /identity/v3/role_assignments?user.id=de239ca2780d41a29fa2e8c7a5df0514&scope.project.id=acb9381780194f6aa6e4f661df696687&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:43.286967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc60c8-b0dd-454b-a1c1-58816aadad86 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:43.324274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:43.324931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:43.325212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:43.325485 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:43.333096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:43.334764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:00:43.347488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:00:43.348553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:00:43.349469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:00:43.349974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 546/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:00:43.354786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:43.461577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ae922-997d-4b50-bfe1-153f71f9fd70 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:43.496693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c1452-779f-47ae-8c1e-a87b6662a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:43.497306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c1452-779f-47ae-8c1e-a87b6662a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:43.497589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c1452-779f-47ae-8c1e-a87b6662a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:43.498097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c1452-779f-47ae-8c1e-a87b6662a0f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:43.499086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c1452-779f-47ae-8c1e-a87b6662a0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:43.499397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c1452-779f-47ae-8c1e-a87b6662a0f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:43.500830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c1452-779f-47ae-8c1e-a87b6662a0f4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:43.512855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 559/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:00:43.519408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:43.565499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:43.620826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98d4bf-6958-4150-aac8-d2b6c381efa6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:43.653408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc0150-08ac-4851-8550-286ebec8dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:43.654395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0150-08ac-4851-8550-286ebec8dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:43.654672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0150-08ac-4851-8550-286ebec8dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:43.654970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0150-08ac-4851-8550-286ebec8dcd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:43.655682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc0150-08ac-4851-8550-286ebec8dcd3 None admin] RBAC: Adding query filter params (role.id=143ac081ca204690a023a1a2c385de76, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=6a4ea803f6e44648bb217abd00e11c3c) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:43.656014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc0150-08ac-4851-8550-286ebec8dcd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:43.657414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc0150-08ac-4851-8550-286ebec8dcd3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:43.663085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 547/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:43 2022] GET /identity/v3/role_assignments?user.id=6a4ea803f6e44648bb217abd00e11c3c&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=143ac081ca204690a023a1a2c385de76 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:45.454588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09024c77-d4ab-41e6-abb3-55993dcc1bbf None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:45.455147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09024c77-d4ab-41e6-abb3-55993dcc1bbf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:45.455147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09024c77-d4ab-41e6-abb3-55993dcc1bbf None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:45.456452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 560/1107] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:45.461662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd217b9b-5af5-419e-af2c-24099283d559 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:45.462095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd217b9b-5af5-419e-af2c-24099283d559 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:45.462244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd217b9b-5af5-419e-af2c-24099283d559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:45.480336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cd217b9b-5af5-419e-af2c-24099283d559 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:45.481500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cd217b9b-5af5-419e-af2c-24099283d559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:45.547985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 548/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:45.815074 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09024c77-d4ab-41e6-abb3-55993dcc1bbf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:45.818830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:45.913602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09024c77-d4ab-41e6-abb3-55993dcc1bbf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:45.946439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcce4b-d339-4bf6-abb4-a5e4b6264c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:45.947057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcce4b-d339-4bf6-abb4-a5e4b6264c89 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:45.947333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcce4b-d339-4bf6-abb4-a5e4b6264c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:45.947604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcce4b-d339-4bf6-abb4-a5e4b6264c89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:45.948700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcce4b-d339-4bf6-abb4-a5e4b6264c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:45.950156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcce4b-d339-4bf6-abb4-a5e4b6264c89 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:45.952237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 561/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:45.958074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd217b9b-5af5-419e-af2c-24099283d559 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:45.991720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:46.053933 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd217b9b-5af5-419e-af2c-24099283d559 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:46.086596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea2117-d1bb-4ad2-bec6-aec120fd24c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:46.087129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea2117-d1bb-4ad2-bec6-aec120fd24c7 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:46.087408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea2117-d1bb-4ad2-bec6-aec120fd24c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:46.087680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea2117-d1bb-4ad2-bec6-aec120fd24c7 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:46.088329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea2117-d1bb-4ad2-bec6-aec120fd24c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:46.089765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea2117-d1bb-4ad2-bec6-aec120fd24c7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:46.110909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 549/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 14:00:45 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:00:48.298146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5187295-378e-4b4c-814b-4101ad5697ed None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:48.298704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5187295-378e-4b4c-814b-4101ad5697ed None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:48.298704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5187295-378e-4b4c-814b-4101ad5697ed None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:48.300010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 562/1111] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:48.305141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2a014-0b35-485c-a2fc-47d797de901c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:48.305521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2a014-0b35-485c-a2fc-47d797de901c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:48.305694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2a014-0b35-485c-a2fc-47d797de901c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:48.323412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-86d2a014-0b35-485c-a2fc-47d797de901c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:48.324587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-86d2a014-0b35-485c-a2fc-47d797de901c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:48.390573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 550/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:48.661020 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5187295-378e-4b4c-814b-4101ad5697ed None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:48.730301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:48.759844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5187295-378e-4b4c-814b-4101ad5697ed None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:48.795244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:48.795860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:48.796140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:48.796409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] PATH_INFO: `/v3/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:48.797814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] RBAC: Authorizing `identity:get_role(role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:48.799174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:48.800471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 563/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:48 2022] GET /identity/v3/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:48.806282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2a014-0b35-485c-a2fc-47d797de901c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:48.830550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:48.905959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2a014-0b35-485c-a2fc-47d797de901c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:48.944283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:48.944837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:48.945118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:48.945392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] PATH_INFO: `/v3/users/35e83588746048078544e437dae2fab3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:48.958735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] RBAC: Authorizing `identity:get_user(user_id=35e83588746048078544e437dae2fab3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:48.960168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:48.961815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 551/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:48 2022] GET /identity/v3/users/35e83588746048078544e437dae2fab3 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:48.967645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:49.064373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbbd1d-1765-47fe-9969-ded7f10e47b2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:49.096335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:49.096918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:49.097192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:49.097461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:49.107716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:49.163031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:49.164465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:49.260168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 564/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:49.266885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:49.344830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:49.363263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eab85-c57c-41a3-958b-8c466dcb6afa None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:49.396698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12d2ff-ff26-4131-a053-7d99bccdc9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:49.397308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d2ff-ff26-4131-a053-7d99bccdc9cd None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:49.397585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d2ff-ff26-4131-a053-7d99bccdc9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:49.397886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d2ff-ff26-4131-a053-7d99bccdc9cd None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:49.398973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12d2ff-ff26-4131-a053-7d99bccdc9cd None admin] RBAC: Authorizing `identity:get_project(project_id=acb9381780194f6aa6e4f661df696687)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:49.400376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12d2ff-ff26-4131-a053-7d99bccdc9cd None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:49.401993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 552/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:00:49 2022] GET /identity/v3/projects/acb9381780194f6aa6e4f661df696687 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:49.408396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:49.448241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:49.504842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92985f12-1cdf-43cf-9ffa-bdf8f1e44cff None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:49.539729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bd7eb-50f5-4907-8f99-21ebe8edeaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:49.540326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bd7eb-50f5-4907-8f99-21ebe8edeaca None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:49.540604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bd7eb-50f5-4907-8f99-21ebe8edeaca None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:49.540874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bd7eb-50f5-4907-8f99-21ebe8edeaca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:49.541589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bd7eb-50f5-4907-8f99-21ebe8edeaca None admin] RBAC: Adding query filter params (role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=acb9381780194f6aa6e4f661df696687, user.id=35e83588746048078544e437dae2fab3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:49.541929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bd7eb-50f5-4907-8f99-21ebe8edeaca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:49.543358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bd7eb-50f5-4907-8f99-21ebe8edeaca None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:49.549250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 565/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:49 2022] GET /identity/v3/role_assignments?user.id=35e83588746048078544e437dae2fab3&scope.project.id=acb9381780194f6aa6e4f661df696687&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:51.661607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b1f07-363e-4ba3-81a3-7270fa96037b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:51.662159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b1f07-363e-4ba3-81a3-7270fa96037b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:51.662245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b1f07-363e-4ba3-81a3-7270fa96037b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:51.663511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 553/1118] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:51.668525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690398d-3772-48ad-b955-91f6f4b5980a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:51.668890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690398d-3772-48ad-b955-91f6f4b5980a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:51.669078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690398d-3772-48ad-b955-91f6f4b5980a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:51.687604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5690398d-3772-48ad-b955-91f6f4b5980a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:51.688773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5690398d-3772-48ad-b955-91f6f4b5980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:51.753558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 566/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:52.031858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b1f07-363e-4ba3-81a3-7270fa96037b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:52.129673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b1f07-363e-4ba3-81a3-7270fa96037b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:52.163744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:52.164355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:52.164631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:52.164901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] PATH_INFO: `/v3/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:52.166293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] RBAC: Authorizing `identity:get_role(role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:52.167634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:52.168925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 554/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:52 2022] GET /identity/v3/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:52.174891 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690398d-3772-48ad-b955-91f6f4b5980a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:52.207699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:52.271212 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690398d-3772-48ad-b955-91f6f4b5980a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:52.304025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:52.304582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:52.304863 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:52.305134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] PATH_INFO: `/v3/users/35e83588746048078544e437dae2fab3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:52.306263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] RBAC: Authorizing `identity:get_user(user_id=35e83588746048078544e437dae2fab3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:52.307680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:52.309224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 567/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:52 2022] GET /identity/v3/users/35e83588746048078544e437dae2fab3 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:52.315616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:52.333274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:52.412250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32801d-4a29-495a-a4ae-fcf9d2c58b4c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:52.444274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86846a0b-11b8-46b0-ae18-86bf135fb5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:52.444883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846a0b-11b8-46b0-ae18-86bf135fb5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:52.445160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846a0b-11b8-46b0-ae18-86bf135fb5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:52.445429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86846a0b-11b8-46b0-ae18-86bf135fb5e0 None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:52.446546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86846a0b-11b8-46b0-ae18-86bf135fb5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=acb9381780194f6aa6e4f661df696687)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:52.447945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86846a0b-11b8-46b0-ae18-86bf135fb5e0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:52.449527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 555/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:00:52 2022] GET /identity/v3/projects/acb9381780194f6aa6e4f661df696687 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:52.455426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:52.553527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253604e-2286-4a81-aefa-6e01714023fb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:52.555103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:52.587989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8557a-eda0-4b90-93d4-e00b3934d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:52.588628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8557a-eda0-4b90-93d4-e00b3934d7ea None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:52.588902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8557a-eda0-4b90-93d4-e00b3934d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:52.589170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8557a-eda0-4b90-93d4-e00b3934d7ea None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687/users/35e83588746048078544e437dae2fab3/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:52.591858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8557a-eda0-4b90-93d4-e00b3934d7ea None admin] RBAC: Authorizing `identity:create_grant(project_id=acb9381780194f6aa6e4f661df696687, user_id=35e83588746048078544e437dae2fab3, role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:52.593284 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8557a-eda0-4b90-93d4-e00b3934d7ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:52.605357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 568/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:00:52 2022] PUT /identity/v3/projects/acb9381780194f6aa6e4f661df696687/users/35e83588746048078544e437dae2fab3/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:00:54.710823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f79628-2411-4748-b39b-f4988aca99d0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:54.711368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f79628-2411-4748-b39b-f4988aca99d0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:54.711368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f79628-2411-4748-b39b-f4988aca99d0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:54.712706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 556/1124] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:54.718045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9fb16-1bba-4b58-b752-2286f29255a0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:54.718466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9fb16-1bba-4b58-b752-2286f29255a0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:54.718583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9fb16-1bba-4b58-b752-2286f29255a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:54.736083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c4f9fb16-1bba-4b58-b752-2286f29255a0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:54.737244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c4f9fb16-1bba-4b58-b752-2286f29255a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:54.802203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 569/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:55.073357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f79628-2411-4748-b39b-f4988aca99d0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:55.156798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:55.173221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f79628-2411-4748-b39b-f4988aca99d0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:55.206800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:55.207419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:55.207700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:55.207977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] PATH_INFO: `/v3/roles/7cd71c8fc5c24e42a4e7574715ef2080` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:55.209355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] RBAC: Authorizing `identity:get_role(role_id=7cd71c8fc5c24e42a4e7574715ef2080)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:55.210748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:55.212053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 557/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:55 2022] GET /identity/v3/roles/7cd71c8fc5c24e42a4e7574715ef2080 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:55.217960 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9fb16-1bba-4b58-b752-2286f29255a0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:55.304238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:55.314017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9fb16-1bba-4b58-b752-2286f29255a0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:55.346470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:55.347023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:55.347298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:55.347568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] PATH_INFO: `/v3/users/35e83588746048078544e437dae2fab3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:55.348666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=35e83588746048078544e437dae2fab3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:55.350106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:55.351646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 570/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:00:55 2022] GET /identity/v3/users/35e83588746048078544e437dae2fab3 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:55.357435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:55.453920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6c58d-fa5e-409b-a5ce-14847a177693 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:55.486193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:55.486775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:55.487049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:55.487318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:55.508624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:55.554303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:55.555733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:55.648701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 558/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:00:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:00:55.654832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:55.749532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf0700-7424-41cd-888f-d4f570120bc4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:55.782634 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74942-fd7d-4420-910d-2e50f6d96a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:55.783261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74942-fd7d-4420-910d-2e50f6d96a1e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:55.783517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74942-fd7d-4420-910d-2e50f6d96a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:55.783787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74942-fd7d-4420-910d-2e50f6d96a1e None admin] PATH_INFO: `/v3/projects/acb9381780194f6aa6e4f661df696687` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:55.784873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74942-fd7d-4420-910d-2e50f6d96a1e None admin] RBAC: Authorizing `identity:get_project(project_id=acb9381780194f6aa6e4f661df696687)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:55.786314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74942-fd7d-4420-910d-2e50f6d96a1e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:55.788058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 571/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:00:55 2022] GET /identity/v3/projects/acb9381780194f6aa6e4f661df696687 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:55.794036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:55.845332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:55.890654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4846a-5996-4cd6-abec-16f58dbbba46 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:55.923193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f15c4d-7017-4f22-a4f5-8cde1fed8821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:55.923713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15c4d-7017-4f22-a4f5-8cde1fed8821 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:55.923987 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15c4d-7017-4f22-a4f5-8cde1fed8821 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:55.924254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15c4d-7017-4f22-a4f5-8cde1fed8821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:55.924957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15c4d-7017-4f22-a4f5-8cde1fed8821 None admin] RBAC: Adding query filter params (role.id=7cd71c8fc5c24e42a4e7574715ef2080, scope.project.id=acb9381780194f6aa6e4f661df696687, user.id=35e83588746048078544e437dae2fab3) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:00:55.925263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15c4d-7017-4f22-a4f5-8cde1fed8821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:55.926689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f15c4d-7017-4f22-a4f5-8cde1fed8821 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:55.932158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 559/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:00:55 2022] GET /identity/v3/role_assignments?user.id=35e83588746048078544e437dae2fab3&scope.project.id=acb9381780194f6aa6e4f661df696687&role.id=7cd71c8fc5c24e42a4e7574715ef2080 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:58.076250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848c2c1-ce52-4d8a-829b-706cda2ebfb5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:58.076850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848c2c1-ce52-4d8a-829b-706cda2ebfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:58.076850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848c2c1-ce52-4d8a-829b-706cda2ebfb5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:58.078197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 572/1131] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:00:58.083259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3ebe-7b0a-444c-acd1-c6829b0ca87b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:58.083687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3ebe-7b0a-444c-acd1-c6829b0ca87b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:58.083811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3ebe-7b0a-444c-acd1-c6829b0ca87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:58.101573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ad8c3ebe-7b0a-444c-acd1-c6829b0ca87b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:00:58.102777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ad8c3ebe-7b0a-444c-acd1-c6829b0ca87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:00:58.167141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 560/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:00:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:00:58.446184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848c2c1-ce52-4d8a-829b-706cda2ebfb5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:58.474692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:58.548459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848c2c1-ce52-4d8a-829b-706cda2ebfb5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:58.581620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed6bc9-465d-44d4-b407-6ecbf3b4a862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:58.582264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed6bc9-465d-44d4-b407-6ecbf3b4a862 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:58.582541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed6bc9-465d-44d4-b407-6ecbf3b4a862 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:58.582812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed6bc9-465d-44d4-b407-6ecbf3b4a862 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:58.583913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed6bc9-465d-44d4-b407-6ecbf3b4a862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:58.585334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed6bc9-465d-44d4-b407-6ecbf3b4a862 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:58.586921 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 573/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:00:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:00:58.592972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c3ebe-7b0a-444c-acd1-c6829b0ca87b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:00:58.637676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:00:58.688896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c3ebe-7b0a-444c-acd1-c6829b0ca87b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:00:58.720743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df4d4b-f1f5-4668-b10f-96c8b16578a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:00:58.721330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df4d4b-f1f5-4668-b10f-96c8b16578a3 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:00:58.721608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df4d4b-f1f5-4668-b10f-96c8b16578a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:00:58.721913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df4d4b-f1f5-4668-b10f-96c8b16578a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:00:58.722546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df4d4b-f1f5-4668-b10f-96c8b16578a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:00:58.723956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df4d4b-f1f5-4668-b10f-96c8b16578a3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:00:58.750475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 561/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:00:58 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:01:00.910134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498aa16-ed1a-4201-9a2d-e6b9d3b6b2cc None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:00.910650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498aa16-ed1a-4201-9a2d-e6b9d3b6b2cc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:00.910650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498aa16-ed1a-4201-9a2d-e6b9d3b6b2cc None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:00.911925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 574/1135] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:01:00.917231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a2a1c-212e-4709-b65d-e991f1f62d91 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:00.917622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a2a1c-212e-4709-b65d-e991f1f62d91 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:00.917815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a2a1c-212e-4709-b65d-e991f1f62d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:00.936714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-809a2a1c-212e-4709-b65d-e991f1f62d91 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:00.937925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-809a2a1c-212e-4709-b65d-e991f1f62d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:01.001612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 562/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:01.271063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498aa16-ed1a-4201-9a2d-e6b9d3b6b2cc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:01.364375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:01.371365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498aa16-ed1a-4201-9a2d-e6b9d3b6b2cc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:01.405503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f883cc-7c50-42dd-b566-a79a8bc5adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:01.406145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f883cc-7c50-42dd-b566-a79a8bc5adcc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:01.406427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f883cc-7c50-42dd-b566-a79a8bc5adcc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:01.406701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f883cc-7c50-42dd-b566-a79a8bc5adcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:01.407802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f883cc-7c50-42dd-b566-a79a8bc5adcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:01.409222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f883cc-7c50-42dd-b566-a79a8bc5adcc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:01.410794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 575/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:01:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:01.416458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a2a1c-212e-4709-b65d-e991f1f62d91 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:01.443407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:01.512804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a2a1c-212e-4709-b65d-e991f1f62d91 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:01.547504 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502e7d6-532e-461c-ad48-2e7cd378b1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:01.548103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502e7d6-532e-461c-ad48-2e7cd378b1d1 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:01.548376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502e7d6-532e-461c-ad48-2e7cd378b1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:01.548641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502e7d6-532e-461c-ad48-2e7cd378b1d1 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:01.549286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502e7d6-532e-461c-ad48-2e7cd378b1d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:01.550723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502e7d6-532e-461c-ad48-2e7cd378b1d1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:01.570947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 563/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 14:01:01 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:01:03.711590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24ed94-7209-47fc-b599-5916b8f1ac99 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:03.712161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24ed94-7209-47fc-b599-5916b8f1ac99 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:03.712161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24ed94-7209-47fc-b599-5916b8f1ac99 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:03.713441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 576/1139] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:03.719116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644f551-e6c5-4776-80b3-d5ead9ecbbd0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:03.719551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644f551-e6c5-4776-80b3-d5ead9ecbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:03.719651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644f551-e6c5-4776-80b3-d5ead9ecbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:03.738025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0644f551-e6c5-4776-80b3-d5ead9ecbbd0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:03.739193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0644f551-e6c5-4776-80b3-d5ead9ecbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:03.803063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 564/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:01:04.070190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24ed94-7209-47fc-b599-5916b8f1ac99 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:04.168581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24ed94-7209-47fc-b599-5916b8f1ac99 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:04.202087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:04.202704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:04.202981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:04.203248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:04.212960 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:01:04.218348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:01:04.219530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:01:04.220667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 577/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:01:04.225425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644f551-e6c5-4776-80b3-d5ead9ecbbd0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:04.240718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:04.323017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644f551-e6c5-4776-80b3-d5ead9ecbbd0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:04.356036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:04.356654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:04.357637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:04.358186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:04.358857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:04.359165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:04.360525 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:04.368292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 565/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:04.374702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:04.399540 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:04.471811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2e106-3fa9-43ee-abbf-21c9d1437480 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:04.504545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:04.505156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:04.505430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:04.505697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] PATH_INFO: `/v3/users/9ec6559dafc34377a17e6d8bb64f32c6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:04.518873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] RBAC: Authorizing `identity:get_user(user_id=9ec6559dafc34377a17e6d8bb64f32c6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:04.520290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:04.521889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 578/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:01:04 2022] GET /identity/v3/users/9ec6559dafc34377a17e6d8bb64f32c6 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:04.527798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:04.603491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:04.625950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4f7b9-6738-4025-8a66-9d09eaef0003 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:04.658784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:04.659380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:04.659656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:04.659926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:04.723831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:04.725253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:04.795806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:04.818359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 566/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:01:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:01:04.824395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:04.905334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:04.923021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc648ce-9c57-4228-b31c-3b72588372e3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:04.956552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48369db5-ee6e-419b-93dc-b90197bbf0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:04.957176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48369db5-ee6e-419b-93dc-b90197bbf0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:04.957454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48369db5-ee6e-419b-93dc-b90197bbf0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:04.957743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48369db5-ee6e-419b-93dc-b90197bbf0d6 None admin] PATH_INFO: `/v3/projects/4772e930acda42808829d00c2eb018c6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:04.958859 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48369db5-ee6e-419b-93dc-b90197bbf0d6 None admin] RBAC: Authorizing `identity:get_project(project_id=4772e930acda42808829d00c2eb018c6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:04.960281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48369db5-ee6e-419b-93dc-b90197bbf0d6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:04.961932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 579/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:01:04 2022] GET /identity/v3/projects/4772e930acda42808829d00c2eb018c6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:04.968472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:05.064997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab110-cfc7-4f0b-bf10-20635b6da8f3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:05.097585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b06498-1087-4ce8-83af-fd8edd5234c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:05.098198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b06498-1087-4ce8-83af-fd8edd5234c0 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:05.098480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b06498-1087-4ce8-83af-fd8edd5234c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:05.098751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b06498-1087-4ce8-83af-fd8edd5234c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:05.099467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b06498-1087-4ce8-83af-fd8edd5234c0 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=4772e930acda42808829d00c2eb018c6, user.id=9ec6559dafc34377a17e6d8bb64f32c6) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:05.099775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b06498-1087-4ce8-83af-fd8edd5234c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:05.101179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b06498-1087-4ce8-83af-fd8edd5234c0 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:05.106384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 567/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:01:04 2022] GET /identity/v3/role_assignments?user.id=9ec6559dafc34377a17e6d8bb64f32c6&scope.project.id=4772e930acda42808829d00c2eb018c6&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:07.187569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4f4af-9e00-47f3-acdd-ff86a7551943 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:07.188132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4f4af-9e00-47f3-acdd-ff86a7551943 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:07.188132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4f4af-9e00-47f3-acdd-ff86a7551943 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:07.189436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 580/1147] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:07.194470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0d086-95eb-406f-ab62-500a6f74d23e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:07.194892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0d086-95eb-406f-ab62-500a6f74d23e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:07.195021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0d086-95eb-406f-ab62-500a6f74d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:07.213980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-26f0d086-95eb-406f-ab62-500a6f74d23e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:07.215144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-26f0d086-95eb-406f-ab62-500a6f74d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:07.247756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:07.279628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 568/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:07.549357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4f4af-9e00-47f3-acdd-ff86a7551943 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:07.647583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4f4af-9e00-47f3-acdd-ff86a7551943 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:07.681274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:07.681918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:07.682195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:07.682462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:07.692886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:01:07.697647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:01:07.698787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:01:07.699788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 581/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:01:07.705125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0d086-95eb-406f-ab62-500a6f74d23e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:07.800428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0d086-95eb-406f-ab62-500a6f74d23e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:07.832757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:07.833349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:07.833630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:07.833938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:07.834600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:07.834903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:07.836251 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:07.843952 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 569/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:01:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:07.849670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:07.862586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:07.948907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacaa7-d703-44ab-9e26-c9478ba6b21f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:07.981062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:07.981811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:07.982197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:07.982364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] PATH_INFO: `/v3/users/9ec6559dafc34377a17e6d8bb64f32c6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:07.983461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] RBAC: Authorizing `identity:get_user(user_id=9ec6559dafc34377a17e6d8bb64f32c6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:07.984874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:07.986437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 582/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:01:07 2022] GET /identity/v3/users/9ec6559dafc34377a17e6d8bb64f32c6 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:07.992171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:08.023454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:08.091162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4aa31-bd10-4576-b426-1d712f5e5720 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:08.123650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e4215-0f49-42ec-b1dc-b729780b6287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:08.124261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e4215-0f49-42ec-b1dc-b729780b6287 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:08.124536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e4215-0f49-42ec-b1dc-b729780b6287 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:08.124806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e4215-0f49-42ec-b1dc-b729780b6287 None admin] PATH_INFO: `/v3/projects/4772e930acda42808829d00c2eb018c6` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:08.125956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e4215-0f49-42ec-b1dc-b729780b6287 None admin] RBAC: Authorizing `identity:get_project(project_id=4772e930acda42808829d00c2eb018c6)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:08.127352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e4215-0f49-42ec-b1dc-b729780b6287 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:08.128937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 570/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:01:07 2022] GET /identity/v3/projects/4772e930acda42808829d00c2eb018c6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:08.134827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:08.213179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:08.236170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd293fc8-0fa1-40ff-ad40-14a42af95f0a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:08.270064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6042206-8368-484b-b743-64776bdaf495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:08.270706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042206-8368-484b-b743-64776bdaf495 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:08.270985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042206-8368-484b-b743-64776bdaf495 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:08.271256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042206-8368-484b-b743-64776bdaf495 None admin] PATH_INFO: `/v3/projects/4772e930acda42808829d00c2eb018c6/users/9ec6559dafc34377a17e6d8bb64f32c6/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:08.273126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6042206-8368-484b-b743-64776bdaf495 None admin] RBAC: Authorizing `identity:create_grant(project_id=4772e930acda42808829d00c2eb018c6, user_id=9ec6559dafc34377a17e6d8bb64f32c6, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:08.274574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6042206-8368-484b-b743-64776bdaf495 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:08.287652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 583/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:01:08 2022] PUT /identity/v3/projects/4772e930acda42808829d00c2eb018c6/users/9ec6559dafc34377a17e6d8bb64f32c6/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:01:10.374430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb7fd9-c6cc-49b2-805d-42d8f2745313 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:10.375070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb7fd9-c6cc-49b2-805d-42d8f2745313 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:10.375070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb7fd9-c6cc-49b2-805d-42d8f2745313 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:10.376274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 571/1154] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:01:10.381428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca42be-55a8-4ffd-a255-1e0b4e9831dc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:10.381917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca42be-55a8-4ffd-a255-1e0b4e9831dc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:10.382018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca42be-55a8-4ffd-a255-1e0b4e9831dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:10.400167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aaca42be-55a8-4ffd-a255-1e0b4e9831dc None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:10.401333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aaca42be-55a8-4ffd-a255-1e0b4e9831dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:10.466365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 584/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:10.738809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb7fd9-c6cc-49b2-805d-42d8f2745313 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:10.834269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:10.838578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbb7fd9-c6cc-49b2-805d-42d8f2745313 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:10.871270 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:10.871878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:10.873020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:10.873299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:10.882928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:01:10.887804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:01:10.889045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:01:10.890274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 572/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:01:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:01:10.894835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca42be-55a8-4ffd-a255-1e0b4e9831dc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:10.960056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:10.994006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca42be-55a8-4ffd-a255-1e0b4e9831dc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:11.025849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:11.026432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:11.026705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:11.026972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:11.027617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:11.027914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:11.030057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:11.037796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 585/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:11.043657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:11.140447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a61c8-cfc8-4977-9ab0-8d8531c674ac None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:11.173978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:11.174583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:11.174860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:11.175129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] PATH_INFO: `/v3/users/9ec6559dafc34377a17e6d8bb64f32c6` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:11.176238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] RBAC: Authorizing `identity:get_user(user_id=9ec6559dafc34377a17e6d8bb64f32c6)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:11.177645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:11.179205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 573/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:01:11 2022] GET /identity/v3/users/9ec6559dafc34377a17e6d8bb64f32c6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:11.185176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:11.281349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca841f-a110-4ba9-95cf-231c39429c3f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:11.307655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:11.314110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:11.314709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:11.314987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:11.315260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:11.380287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:11.381715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:11.476924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 586/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:01:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:01:11.483651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:11.530153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:11.609801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbee96-1dd9-4d61-8160-e30311e001fe None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:11.642645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac090b66-78fd-46f1-9b15-532168f6a79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:11.643255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac090b66-78fd-46f1-9b15-532168f6a79e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:11.643531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac090b66-78fd-46f1-9b15-532168f6a79e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:11.643800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac090b66-78fd-46f1-9b15-532168f6a79e None admin] PATH_INFO: `/v3/projects/4772e930acda42808829d00c2eb018c6` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:11.644887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac090b66-78fd-46f1-9b15-532168f6a79e None admin] RBAC: Authorizing `identity:get_project(project_id=4772e930acda42808829d00c2eb018c6)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:11.646327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac090b66-78fd-46f1-9b15-532168f6a79e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:11.647933 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 574/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:01:11 2022] GET /identity/v3/projects/4772e930acda42808829d00c2eb018c6 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:11.655032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:11.683767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:11.750018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e64778-8bc4-40cb-b0af-820ac0c84d98 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:11.782689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d738a-a2f0-434e-bb41-624e625eb3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:11.783271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d738a-a2f0-434e-bb41-624e625eb3fc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:11.783547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d738a-a2f0-434e-bb41-624e625eb3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:11.783816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d738a-a2f0-434e-bb41-624e625eb3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:11.784518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d738a-a2f0-434e-bb41-624e625eb3fc None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=4772e930acda42808829d00c2eb018c6, user.id=9ec6559dafc34377a17e6d8bb64f32c6) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:11.784821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d738a-a2f0-434e-bb41-624e625eb3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:11.786253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d738a-a2f0-434e-bb41-624e625eb3fc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:11.791722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 587/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:01:11 2022] GET /identity/v3/role_assignments?user.id=9ec6559dafc34377a17e6d8bb64f32c6&scope.project.id=4772e930acda42808829d00c2eb018c6&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:13.931863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635b66-551c-434f-be3a-59027e3a3970 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:13.932386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635b66-551c-434f-be3a-59027e3a3970 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:13.932386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635b66-551c-434f-be3a-59027e3a3970 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:13.933625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 575/1162] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:13.939824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2417-df46-4dcf-b97a-bce4a6b7aac8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:13.940208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2417-df46-4dcf-b97a-bce4a6b7aac8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:13.940378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd2417-df46-4dcf-b97a-bce4a6b7aac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:13.958130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4acd2417-df46-4dcf-b97a-bce4a6b7aac8 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:13.959304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4acd2417-df46-4dcf-b97a-bce4a6b7aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:14.024008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 588/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:14.302424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635b66-551c-434f-be3a-59027e3a3970 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:14.393887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:14.400638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635b66-551c-434f-be3a-59027e3a3970 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:14.434146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:14.434843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:14.435125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:14.435407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:14.499237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:14.500661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:14.589788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:14.597144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 576/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:01:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:01:14.603004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2417-df46-4dcf-b97a-bce4a6b7aac8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:14.626662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:14.697882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd2417-df46-4dcf-b97a-bce4a6b7aac8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:14.730150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b169c-e3a3-4e6a-b90a-23102fe2007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:14.730758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b169c-e3a3-4e6a-b90a-23102fe2007b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:14.731032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b169c-e3a3-4e6a-b90a-23102fe2007b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:14.731298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b169c-e3a3-4e6a-b90a-23102fe2007b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:14.737658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b169c-e3a3-4e6a-b90a-23102fe2007b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:14.739103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b169c-e3a3-4e6a-b90a-23102fe2007b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-053b169c-e3a3-4e6a-b90a-23102fe2007b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:01:14.750714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:01:14.751948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:01:14.752830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 589/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:01:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:01:14.757655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:14.852693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c0715-60b8-4ee5-a4e1-e401abc97f48 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:14.885191 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7607a-b175-4e98-b935-2dc748b04eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:14.885809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7607a-b175-4e98-b935-2dc748b04eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:14.886091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7607a-b175-4e98-b935-2dc748b04eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:14.886358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7607a-b175-4e98-b935-2dc748b04eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:14.886984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7607a-b175-4e98-b935-2dc748b04eb7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:14.887282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7607a-b175-4e98-b935-2dc748b04eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:14.888762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7607a-b175-4e98-b935-2dc748b04eb7 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:14.895610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 577/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Sep 13 14:01:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:17.167176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e051f-757b-4276-b4a0-40fc88e9ef1d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:17.167732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e051f-757b-4276-b4a0-40fc88e9ef1d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:17.167732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e051f-757b-4276-b4a0-40fc88e9ef1d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:17.169060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 590/1167] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:17.174941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99588246-d232-444a-8dec-cc015719f059 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:17.175385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99588246-d232-444a-8dec-cc015719f059 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:17.175497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99588246-d232-444a-8dec-cc015719f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:17.193203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-99588246-d232-444a-8dec-cc015719f059 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:17.194398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-99588246-d232-444a-8dec-cc015719f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:17.248100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:17.259226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 578/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:17.536153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e051f-757b-4276-b4a0-40fc88e9ef1d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:17.619079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:17.636057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e051f-757b-4276-b4a0-40fc88e9ef1d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:17.670132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dcd75-4163-40fd-9894-b0535a9858e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:17.670730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcd75-4163-40fd-9894-b0535a9858e1 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:17.671006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcd75-4163-40fd-9894-b0535a9858e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:17.671273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcd75-4163-40fd-9894-b0535a9858e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:17.671935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dcd75-4163-40fd-9894-b0535a9858e1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:17.673290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dcd75-4163-40fd-9894-b0535a9858e1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:17.688881 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 591/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Tue Sep 13 14:01:17 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:01:19.896961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22493e64-e240-435a-bcf7-f9764ab6a847 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:19.897477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22493e64-e240-435a-bcf7-f9764ab6a847 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:19.897477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22493e64-e240-435a-bcf7-f9764ab6a847 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:19.898807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 579/1170] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:19.904971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405c817-c1f0-4ccc-8127-8c463e53aa99 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:19.905392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405c817-c1f0-4ccc-8127-8c463e53aa99 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:19.905494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405c817-c1f0-4ccc-8127-8c463e53aa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:19.924380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7405c817-c1f0-4ccc-8127-8c463e53aa99 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:19.926338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7405c817-c1f0-4ccc-8127-8c463e53aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:19.991105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 592/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:20.257446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22493e64-e240-435a-bcf7-f9764ab6a847 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:20.356736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22493e64-e240-435a-bcf7-f9764ab6a847 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:20.389895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154eb24-ba36-4f70-983c-3c22198426fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:20.390510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154eb24-ba36-4f70-983c-3c22198426fa None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:20.390788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154eb24-ba36-4f70-983c-3c22198426fa None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:20.391058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154eb24-ba36-4f70-983c-3c22198426fa None admin] PATH_INFO: `/v3/domains/fe97ce4bf9c44f9084cd25d7459efcaf` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:20.400969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154eb24-ba36-4f70-983c-3c22198426fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe97ce4bf9c44f9084cd25d7459efcaf)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:20.402418 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154eb24-ba36-4f70-983c-3c22198426fa None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:20.404951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 580/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Sep 13 14:01:20 2022] GET /identity/v3/domains/fe97ce4bf9c44f9084cd25d7459efcaf => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:20.410809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405c817-c1f0-4ccc-8127-8c463e53aa99 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:20.482720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:20.508217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405c817-c1f0-4ccc-8127-8c463e53aa99 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:20.542987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774b109-1573-4674-86f8-e7fa0678b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:20.543594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774b109-1573-4674-86f8-e7fa0678b1fb None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:20.544365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774b109-1573-4674-86f8-e7fa0678b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:20.544632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774b109-1573-4674-86f8-e7fa0678b1fb None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:20.545270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774b109-1573-4674-86f8-e7fa0678b1fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:20.546720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3774b109-1573-4674-86f8-e7fa0678b1fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:20.572421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 593/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:01:20 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:01:22.752851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c415f-b714-4cc4-a841-08a127d91b5e None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:22.753377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c415f-b714-4cc4-a841-08a127d91b5e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:22.753377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c415f-b714-4cc4-a841-08a127d91b5e None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:22.754748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 581/1174] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:22.759888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2c3ba-2557-45d4-b9c0-bfc0c0ebdd0b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:22.760268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2c3ba-2557-45d4-b9c0-bfc0c0ebdd0b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:22.760443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2c3ba-2557-45d4-b9c0-bfc0c0ebdd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:22.778621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-86d2c3ba-2557-45d4-b9c0-bfc0c0ebdd0b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:22.779986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-86d2c3ba-2557-45d4-b9c0-bfc0c0ebdd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:22.844671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 594/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:23.113910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c415f-b714-4cc4-a841-08a127d91b5e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:23.164500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:23.214544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c415f-b714-4cc4-a841-08a127d91b5e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:23.246699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774709dc-d101-499e-94e2-79491df04994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:23.247909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774709dc-d101-499e-94e2-79491df04994 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:23.248196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774709dc-d101-499e-94e2-79491df04994 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:23.248469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774709dc-d101-499e-94e2-79491df04994 None admin] PATH_INFO: `/v3/domains/fe97ce4bf9c44f9084cd25d7459efcaf` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:23.249583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774709dc-d101-499e-94e2-79491df04994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe97ce4bf9c44f9084cd25d7459efcaf)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:23.251025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774709dc-d101-499e-94e2-79491df04994 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:23.252558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 582/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Sep 13 14:01:23 2022] GET /identity/v3/domains/fe97ce4bf9c44f9084cd25d7459efcaf => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:23.258227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2c3ba-2557-45d4-b9c0-bfc0c0ebdd0b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:23.313511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:23.355236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2c3ba-2557-45d4-b9c0-bfc0c0ebdd0b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:23.388568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbe2d3-8bd8-4564-b1a4-1b9f6ec6ed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:23.389097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbe2d3-8bd8-4564-b1a4-1b9f6ec6ed6c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:23.389374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbe2d3-8bd8-4564-b1a4-1b9f6ec6ed6c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:23.389644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbe2d3-8bd8-4564-b1a4-1b9f6ec6ed6c None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:23.390348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbe2d3-8bd8-4564-b1a4-1b9f6ec6ed6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:23.391757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbe2d3-8bd8-4564-b1a4-1b9f6ec6ed6c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:23.412505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 595/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 14:01:23 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:01:25.573573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946db63-9b1c-42a4-a7b4-c786d5c098b8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:25.574134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946db63-9b1c-42a4-a7b4-c786d5c098b8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:25.574134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946db63-9b1c-42a4-a7b4-c786d5c098b8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:25.575498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 583/1178] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:25.580623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da629f1-8f8d-4263-bd98-d28e01dee2db None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:25.581010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da629f1-8f8d-4263-bd98-d28e01dee2db None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:25.581165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da629f1-8f8d-4263-bd98-d28e01dee2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:25.598837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9da629f1-8f8d-4263-bd98-d28e01dee2db None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:25.599995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9da629f1-8f8d-4263-bd98-d28e01dee2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:25.664515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 596/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:25.933588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946db63-9b1c-42a4-a7b4-c786d5c098b8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:26.031064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946db63-9b1c-42a4-a7b4-c786d5c098b8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:26.047817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:26.064317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:26.064926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:26.065201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:26.065470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:26.075179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:01:26.079928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:01:26.081156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:01:26.082258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 584/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:01:26.086924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da629f1-8f8d-4263-bd98-d28e01dee2db None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:26.129037 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:26.186671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da629f1-8f8d-4263-bd98-d28e01dee2db None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:26.219449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:26.220053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:26.220336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:26.220611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:26.221274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:26.221579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:26.222974 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:26.230763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 597/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:01:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:26.236827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:26.339402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09d700-66e6-44c7-99b9-ab5822b94428 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:26.372900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:26.373515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:26.373816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:26.374090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] PATH_INFO: `/v3/users/d7d65cdf635a4f87a4a47c977db8f933` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:26.386625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=d7d65cdf635a4f87a4a47c977db8f933)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:26.388607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:26.390195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 585/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:01:26 2022] GET /identity/v3/users/d7d65cdf635a4f87a4a47c977db8f933 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:26.396025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:26.494279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f818f35-05b1-4e01-8b12-96617e0934f7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:26.496512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:26.529330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:26.529970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:26.530250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:26.530519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:26.596424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:26.597902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:26.691787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 598/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:01:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:01:26.697697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:26.731812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:26.794470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765a428-aa77-4746-92c4-6e5e56a3f9a9 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:26.827736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b69cc-771a-4fd4-902f-3a5d236544e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:26.828347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b69cc-771a-4fd4-902f-3a5d236544e5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:26.828627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b69cc-771a-4fd4-902f-3a5d236544e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:26.828901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b69cc-771a-4fd4-902f-3a5d236544e5 None admin] PATH_INFO: `/v3/projects/40fbe7e4700d4555b6080c1a6c52e759` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:26.830029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b69cc-771a-4fd4-902f-3a5d236544e5 None admin] RBAC: Authorizing `identity:get_project(project_id=40fbe7e4700d4555b6080c1a6c52e759)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:26.831446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b69cc-771a-4fd4-902f-3a5d236544e5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:26.833044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 586/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:01:26 2022] GET /identity/v3/projects/40fbe7e4700d4555b6080c1a6c52e759 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:26.839137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:26.842427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:26.935823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812e7c4-d259-4eb9-803f-d1e7299f8fc6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:26.969022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5c60a-9de4-43d0-a419-8598f97c2704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:26.969613 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5c60a-9de4-43d0-a419-8598f97c2704 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:26.969919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5c60a-9de4-43d0-a419-8598f97c2704 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:26.970191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5c60a-9de4-43d0-a419-8598f97c2704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:26.970904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5c60a-9de4-43d0-a419-8598f97c2704 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=40fbe7e4700d4555b6080c1a6c52e759, user.id=d7d65cdf635a4f87a4a47c977db8f933) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:26.971210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5c60a-9de4-43d0-a419-8598f97c2704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:26.972625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5c60a-9de4-43d0-a419-8598f97c2704 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:26.977880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 599/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:01:26 2022] GET /identity/v3/role_assignments?user.id=d7d65cdf635a4f87a4a47c977db8f933&scope.project.id=40fbe7e4700d4555b6080c1a6c52e759&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:29.075643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb61a6-db96-42f0-8886-f7f92d58d810 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:29.076172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb61a6-db96-42f0-8886-f7f92d58d810 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:29.076172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb61a6-db96-42f0-8886-f7f92d58d810 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:29.077501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 587/1186] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:29.082577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ddfa5-c619-4f09-97f0-b78098f4c709 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:29.082982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ddfa5-c619-4f09-97f0-b78098f4c709 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:29.083132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ddfa5-c619-4f09-97f0-b78098f4c709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:29.102392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-557ddfa5-c619-4f09-97f0-b78098f4c709 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:29.103565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-557ddfa5-c619-4f09-97f0-b78098f4c709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:29.163119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:29.169256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 600/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:29.438950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb61a6-db96-42f0-8886-f7f92d58d810 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:29.538932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb61a6-db96-42f0-8886-f7f92d58d810 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:29.541212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:29.572556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:29.573181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:29.573461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:29.573759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:29.584311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:01:29.589109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:01:29.590235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:01:29.591972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 588/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:01:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:01:29.596798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ddfa5-c619-4f09-97f0-b78098f4c709 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:29.693332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ddfa5-c619-4f09-97f0-b78098f4c709 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:29.726519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:29.727113 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:29.727390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:29.727662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:29.728317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:29.728615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:29.730007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:29.737882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 601/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:29.743625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:29.841700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303ddf6-2031-424d-946b-546c4517332b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:29.874820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:29.875429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:29.875708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:29.875983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] PATH_INFO: `/v3/users/d7d65cdf635a4f87a4a47c977db8f933` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:29.877090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] RBAC: Authorizing `identity:get_user(user_id=d7d65cdf635a4f87a4a47c977db8f933)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:29.878535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:29.880110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 589/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:01:29 2022] GET /identity/v3/users/d7d65cdf635a4f87a4a47c977db8f933 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:29.886029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:29.943507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:29.982660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dedca-3ad1-4a37-a530-f2fc4966403b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:30.015434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58fb98-bed0-4a63-ab47-75ad4f7889ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:30.016044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fb98-bed0-4a63-ab47-75ad4f7889ff None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:30.016322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fb98-bed0-4a63-ab47-75ad4f7889ff None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:30.016594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fb98-bed0-4a63-ab47-75ad4f7889ff None admin] PATH_INFO: `/v3/projects/40fbe7e4700d4555b6080c1a6c52e759` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:30.017688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fb98-bed0-4a63-ab47-75ad4f7889ff None admin] RBAC: Authorizing `identity:get_project(project_id=40fbe7e4700d4555b6080c1a6c52e759)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:30.019130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fb98-bed0-4a63-ab47-75ad4f7889ff None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:30.020722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 602/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:01:29 2022] GET /identity/v3/projects/40fbe7e4700d4555b6080c1a6c52e759 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:30.026564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:30.057678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:30.123268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560602d-dccb-4651-a646-0d0d7ab74655 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:30.155292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280401c-2369-4dec-ae80-e985b766ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:30.155931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280401c-2369-4dec-ae80-e985b766ed64 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:30.156207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280401c-2369-4dec-ae80-e985b766ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:30.156476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280401c-2369-4dec-ae80-e985b766ed64 None admin] PATH_INFO: `/v3/projects/40fbe7e4700d4555b6080c1a6c52e759/users/d7d65cdf635a4f87a4a47c977db8f933/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:30.158350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280401c-2369-4dec-ae80-e985b766ed64 None admin] RBAC: Authorizing `identity:create_grant(project_id=40fbe7e4700d4555b6080c1a6c52e759, user_id=d7d65cdf635a4f87a4a47c977db8f933, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:30.159760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280401c-2369-4dec-ae80-e985b766ed64 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:30.172351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 590/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:01:30 2022] PUT /identity/v3/projects/40fbe7e4700d4555b6080c1a6c52e759/users/d7d65cdf635a4f87a4a47c977db8f933/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:01:32.289597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880987a-821a-45ff-8652-daadfe0c0d20 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:32.290189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880987a-821a-45ff-8652-daadfe0c0d20 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:32.290189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880987a-821a-45ff-8652-daadfe0c0d20 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:32.291508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 603/1193] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:01:32.297207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65b80b-767d-4bb3-ae99-281f905e42ec None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:32.297663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65b80b-767d-4bb3-ae99-281f905e42ec None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:32.297806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65b80b-767d-4bb3-ae99-281f905e42ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:32.316241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1c65b80b-767d-4bb3-ae99-281f905e42ec None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:01:32.317402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1c65b80b-767d-4bb3-ae99-281f905e42ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:01:32.381980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 591/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:01:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:01:32.655113 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880987a-821a-45ff-8652-daadfe0c0d20 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:32.754147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880987a-821a-45ff-8652-daadfe0c0d20 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:32.779739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:32.787523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:32.788130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:32.788406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:32.788677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:32.799030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:01:32.803776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:01:32.804895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:01:32.806147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 604/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:01:32.811298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65b80b-767d-4bb3-ae99-281f905e42ec None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:32.834750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:32.907663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65b80b-767d-4bb3-ae99-281f905e42ec None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:32.939921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:32.940503 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:32.940780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:32.941049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:32.941694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:32.942018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:32.943358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:32.951527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 592/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:32.957356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:33.053259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190e403-65de-4d9d-80d2-2c905ddf70d8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:33.085816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:33.086422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:33.086695 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:33.086962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] PATH_INFO: `/v3/users/d7d65cdf635a4f87a4a47c977db8f933` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:33.088051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] RBAC: Authorizing `identity:get_user(user_id=d7d65cdf635a4f87a4a47c977db8f933)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:33.089465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:33.091292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 605/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Sep 13 14:01:32 2022] GET /identity/v3/users/d7d65cdf635a4f87a4a47c977db8f933 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:01:33.097345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:33.184541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:33.200482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1992c1e-33f8-4678-93e7-31877c3109c6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:33.232723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:33.233316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:33.233599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:33.233901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:33.306891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:33.308319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:33.386599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:33.403216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 593/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:01:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:01:33.409439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:33.463520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:33.506566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13631ec2-4a95-4f92-8670-2c92c74ba94e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:33.540050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8e674-c7fd-4e07-9860-c9a550e9c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:33.540671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8e674-c7fd-4e07-9860-c9a550e9c479 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:33.540947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8e674-c7fd-4e07-9860-c9a550e9c479 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:33.541217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8e674-c7fd-4e07-9860-c9a550e9c479 None admin] PATH_INFO: `/v3/projects/40fbe7e4700d4555b6080c1a6c52e759` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:33.542343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8e674-c7fd-4e07-9860-c9a550e9c479 None admin] RBAC: Authorizing `identity:get_project(project_id=40fbe7e4700d4555b6080c1a6c52e759)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:33.543797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8e674-c7fd-4e07-9860-c9a550e9c479 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:33.545407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 606/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Sep 13 14:01:33 2022] GET /identity/v3/projects/40fbe7e4700d4555b6080c1a6c52e759 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:33.551513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:01:33.654762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb6d4c-d601-4a67-a9c8-a4747afb6de3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:01:33.687439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ee6e9-ab9f-4b5e-8cf3-98498e69bdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:01:33.688027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ee6e9-ab9f-4b5e-8cf3-98498e69bdb1 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:01:33.688306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ee6e9-ab9f-4b5e-8cf3-98498e69bdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:01:33.688577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ee6e9-ab9f-4b5e-8cf3-98498e69bdb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:01:33.689290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ee6e9-ab9f-4b5e-8cf3-98498e69bdb1 None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=40fbe7e4700d4555b6080c1a6c52e759, user.id=d7d65cdf635a4f87a4a47c977db8f933) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:01:33.689594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ee6e9-ab9f-4b5e-8cf3-98498e69bdb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:01:33.691035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ee6e9-ab9f-4b5e-8cf3-98498e69bdb1 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:01:33.696556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 594/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:01:33 2022] GET /identity/v3/role_assignments?user.id=d7d65cdf635a4f87a4a47c977db8f933&scope.project.id=40fbe7e4700d4555b6080c1a6c52e759&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:01:43.535849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=38 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:01:43.694074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=76 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:29.674708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53692878-a899-4993-953f-04d31e33283e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:29.675320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53692878-a899-4993-953f-04d31e33283e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:29.675320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53692878-a899-4993-953f-04d31e33283e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:29.676966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 607/1201] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:02:29.683471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef5276-3821-47be-bd21-050106563bf4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:29.683929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef5276-3821-47be-bd21-050106563bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:29.684108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef5276-3821-47be-bd21-050106563bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:29.711531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dbef5276-3821-47be-bd21-050106563bf4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:02:29.713046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dbef5276-3821-47be-bd21-050106563bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:02:29.803770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 595/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:02:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:02:30.109925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53692878-a899-4993-953f-04d31e33283e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:30.253718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53692878-a899-4993-953f-04d31e33283e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:30.295120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:30.295804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:30.296086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:30.296365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:30.309441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:30.310950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:30.332128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:30.333436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:30.334775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:30.334775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:30.334775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:30.334775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 608/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:30.336228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef5276-3821-47be-bd21-050106563bf4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:30.468968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef5276-3821-47be-bd21-050106563bf4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:30.501339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:30.512677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168455ce-318e-4b42-9e69-568ea713a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:30.513281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168455ce-318e-4b42-9e69-568ea713a3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:30.513603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168455ce-318e-4b42-9e69-568ea713a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:30.513936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168455ce-318e-4b42-9e69-568ea713a3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:30.514592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168455ce-318e-4b42-9e69-568ea713a3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:30.514897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168455ce-318e-4b42-9e69-568ea713a3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:30.516322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168455ce-318e-4b42-9e69-568ea713a3e6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:30.533764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 596/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:30.536364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:30.609262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:30.667565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecd70f-beb7-47d3-a260-ae04601519a6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:30.704633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b439bd-799d-4798-8887-bc2b7dc3947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:30.705532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b439bd-799d-4798-8887-bc2b7dc3947c None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:30.705532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b439bd-799d-4798-8887-bc2b7dc3947c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:30.705904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b439bd-799d-4798-8887-bc2b7dc3947c None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:30.706458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b439bd-799d-4798-8887-bc2b7dc3947c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:30.707898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b439bd-799d-4798-8887-bc2b7dc3947c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:30.734619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 609/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Sep 13 14:02:30 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:02:33.182411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b8bcf-e351-42ff-956b-b0bcffa5048a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:33.183164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b8bcf-e351-42ff-956b-b0bcffa5048a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:33.183164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b8bcf-e351-42ff-956b-b0bcffa5048a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:33.184676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 597/1206] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:02:33.190737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca43a-486f-4446-924c-cef9c79c78fe None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:33.191466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca43a-486f-4446-924c-cef9c79c78fe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:33.191466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ca43a-486f-4446-924c-cef9c79c78fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:33.213096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-267ca43a-486f-4446-924c-cef9c79c78fe None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:02:33.214490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-267ca43a-486f-4446-924c-cef9c79c78fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:02:33.308372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 610/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:02:33.618127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b8bcf-e351-42ff-956b-b0bcffa5048a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:33.742085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b8bcf-e351-42ff-956b-b0bcffa5048a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:33.783961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:33.784512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:33.784790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:33.785061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:33.798613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:02:33.803751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:33.804720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:33.806199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 598/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 14:02:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:33.810921 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca43a-486f-4446-924c-cef9c79c78fe None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:33.858405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:33.924249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ca43a-486f-4446-924c-cef9c79c78fe None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:33.958643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:33.959318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:33.959550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:33.959819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:33.960487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:33.960761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:33.962238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:33.971339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 611/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 14:02:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:33.977568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:34.040206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:34.104931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46da410-99e5-47f9-ac8e-8680ad97ce8d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:34.153719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:34.154661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:34.154945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:34.155225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:34.171456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:34.173291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:34.192573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:34.195646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:34.197381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:34.197381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:34.197381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:34.201589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 599/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:34.201925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:34.320032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:34.325121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773643f-cebe-4f3d-8e59-0e072bcf87cf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:34.365329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:34.366041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:34.368713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:34.368713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:34.368713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:34.368713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:34.369036 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:34.383002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 612/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:34.389996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:34.514618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b711e39-70c8-481c-be0a-d52e5c588d38 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:34.516558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:34.555921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:34.556610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:34.556915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:34.557263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:34.567283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:34.568888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:02:34.583925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:34.585132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:34.586385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:34.590667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 600/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:34.591557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:34.733830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32b871-e65e-42e2-b659-ae5781711390 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:34.775503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:34.776122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:34.776987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:34.777295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:34.778024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:34.778332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:34.780064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:34.800244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 613/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 13 14:02:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:02:34.805831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:34.949472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167dfa1-b9e0-48e5-b74c-cc6d8f91919c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:34.988382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:34.988986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:34.989265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:34.989533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:35.024108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:35.074420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:35.076120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:35.208261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 601/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:02:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:02:35.209463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:35.230512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:35.327124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df127296-7178-43fc-a9b4-19916b154fe8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:35.364170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:35.364832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:35.365110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:35.365428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:35.376644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:35.378238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:35.395583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:35.396879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:35.397893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:35.397893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:35.397893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:35.397893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 614/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:02:35.403381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:35.473349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:35.523578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbc282-2c6c-4e0c-9a2c-fec8cf2e10a6 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:35.566069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:35.566813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:35.567155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:35.567481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:35.568346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:35.568653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:35.570366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:35.583873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 602/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:02:35.589955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:35.687432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:35.726263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e992c2-df74-43bc-b569-59c7f0c20176 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:35.770709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:35.771490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:35.771796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:35.772058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:35.780760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:35.782493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:02:35.795836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:35.797125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:35.798423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 615/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:02:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:35.804044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:35.919871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba17b5e-732f-4272-86d8-497cd909da82 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:35.943725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:35.956360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1f92b-7a38-4f4e-b56a-114ecf860309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:35.956977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1f92b-7a38-4f4e-b56a-114ecf860309 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:35.957374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1f92b-7a38-4f4e-b56a-114ecf860309 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:35.957719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1f92b-7a38-4f4e-b56a-114ecf860309 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:35.958706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1f92b-7a38-4f4e-b56a-114ecf860309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:35.959008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1f92b-7a38-4f4e-b56a-114ecf860309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:35.960573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1f92b-7a38-4f4e-b56a-114ecf860309 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:35.973747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 603/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:02:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:35.979893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:36.105385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c47bcb-e38a-4ffd-bf20-c3f772e1ada3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:36.147240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:36.156084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad64ccc-dea5-4c50-adf1-9c77a5fdd03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:36.156845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64ccc-dea5-4c50-adf1-9c77a5fdd03a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:36.157315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64ccc-dea5-4c50-adf1-9c77a5fdd03a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:36.157569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64ccc-dea5-4c50-adf1-9c77a5fdd03a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:36.158521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad64ccc-dea5-4c50-adf1-9c77a5fdd03a None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=d09c0b02c24f4c5db5ed35fdee2589b5) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:36.158916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad64ccc-dea5-4c50-adf1-9c77a5fdd03a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:36.160545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad64ccc-dea5-4c50-adf1-9c77a5fdd03a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:36.167829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 616/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:02:35 2022] GET /identity/v3/role_assignments?user.id=d09c0b02c24f4c5db5ed35fdee2589b5&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:38.554285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def1055-36b6-4d83-9809-9e5054b186e9 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:38.554831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def1055-36b6-4d83-9809-9e5054b186e9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:38.554991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def1055-36b6-4d83-9809-9e5054b186e9 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:38.556595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 604/1220] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:02:38.562771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24480076-ecbc-42f7-86d8-4ae2042b06bd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:38.563247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24480076-ecbc-42f7-86d8-4ae2042b06bd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:38.563518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24480076-ecbc-42f7-86d8-4ae2042b06bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:38.583241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-24480076-ecbc-42f7-86d8-4ae2042b06bd None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:02:38.584566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-24480076-ecbc-42f7-86d8-4ae2042b06bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:02:38.669926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 617/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:02:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 14:02:38.944882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def1055-36b6-4d83-9809-9e5054b186e9 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:39.060691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def1055-36b6-4d83-9809-9e5054b186e9 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:39.104550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:39.105580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:39.105900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:39.106176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:39.117613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:02:39.122560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:39.123563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:39.124490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 605/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 14:02:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:39.131325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24480076-ecbc-42f7-86d8-4ae2042b06bd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:39.247148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24480076-ecbc-42f7-86d8-4ae2042b06bd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:39.273907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:39.285128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:39.285770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:39.286049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:39.286399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:39.287056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:39.287357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:39.288745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:39.297778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 618/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 14:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:39.303667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:39.354766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:39.420149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71bade-be5e-40e0-9380-d75a8624874f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:39.457710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:39.458352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:39.458628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:39.458896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:39.469278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:39.470732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:39.486795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:39.487964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:39.488889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:39.488889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:39.488889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:39.488889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 606/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:39.494210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:39.601603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25adb15-634f-4d9e-a664-e0aac453edd7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:39.638108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:39.638785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:39.639115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:39.639442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:39.640250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:39.640559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:39.642018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:39.656555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 619/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:39.662850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:39.769582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:39.777962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc9f4-790e-417c-bd05-1c25fe2859bf None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:39.816590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:39.817374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:39.817656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:39.817961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:39.826463 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:39.828279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:02:39.840528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:39.841709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:39.842734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 607/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:39.848152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:39.898237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:39.954875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819de90-934d-4b46-a998-c6a28cd9fa34 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:39.990240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:39.990943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:39.991211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:39.991469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:39.992290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:39.992601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:39.994067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:40.012527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 620/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 13 14:02:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:40.018666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:40.129021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e17e5-7127-4cd5-aaee-d2a205bde38d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:40.167223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:40.168075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:40.168418 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:40.168793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:40.183285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:40.185185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:40.204420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:40.205662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:40.206616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:40.206616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:40.206616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:40.206616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 608/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:40.212220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:40.327648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9415d-d9f4-4baf-9de7-25f05e9c77ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:40.329994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:40.363005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:40.363608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:40.363884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:40.364153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:40.364796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:40.365095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:40.366521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:40.384001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 621/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:02:40.385420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:40.397491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:40.495743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb7b0c-84bc-4e7f-9fd1-1de046f2f6af None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:40.543014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:40.543657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:40.543939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:40.544215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:40.552563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:40.554226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:02:40.569115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:40.570232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:40.571140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 609/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:40.577266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:40.700439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced07e-e358-47d8-a29c-7fb8dce2b481 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:40.740679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590671f-0d4b-459e-9c06-ff950e6e93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:40.741326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590671f-0d4b-459e-9c06-ff950e6e93df None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:40.741636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590671f-0d4b-459e-9c06-ff950e6e93df None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:40.741943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590671f-0d4b-459e-9c06-ff950e6e93df None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:40.742710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590671f-0d4b-459e-9c06-ff950e6e93df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:40.743018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590671f-0d4b-459e-9c06-ff950e6e93df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:40.744452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590671f-0d4b-459e-9c06-ff950e6e93df None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:40.758048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 622/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:40.764556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:40.836406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:40.878620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91f548-49d2-46ab-88d9-90dbdfd5cdef None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:40.913875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf4f25-8beb-4a3f-aade-f29ac7922ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:40.914539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf4f25-8beb-4a3f-aade-f29ac7922ed9 None admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:40.914824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf4f25-8beb-4a3f-aade-f29ac7922ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:40.915102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf4f25-8beb-4a3f-aade-f29ac7922ed9 None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/d09c0b02c24f4c5db5ed35fdee2589b5/roles/adad88d8a75747a381a2b923728d27f8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:40.930548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf4f25-8beb-4a3f-aade-f29ac7922ed9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=d09c0b02c24f4c5db5ed35fdee2589b5, role_id=adad88d8a75747a381a2b923728d27f8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:40.931985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf4f25-8beb-4a3f-aade-f29ac7922ed9 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:40.944932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 610/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:02:40 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/d09c0b02c24f4c5db5ed35fdee2589b5/roles/adad88d8a75747a381a2b923728d27f8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:02:43.199393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a756999-0c09-4f2e-9405-05dab9e664c7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:43.200041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a756999-0c09-4f2e-9405-05dab9e664c7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:43.200041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a756999-0c09-4f2e-9405-05dab9e664c7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:43.201476 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 623/1233] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:02:43.207465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f8a78-8d62-4302-825e-b29122ad4d3b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:43.207959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f8a78-8d62-4302-825e-b29122ad4d3b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:43.208059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f8a78-8d62-4302-825e-b29122ad4d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:43.229205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d19f8a78-8d62-4302-825e-b29122ad4d3b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:02:43.231256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d19f8a78-8d62-4302-825e-b29122ad4d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:02:43.306130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 611/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:02:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:02:43.615645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a756999-0c09-4f2e-9405-05dab9e664c7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:43.644154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:43.736478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a756999-0c09-4f2e-9405-05dab9e664c7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:43.774042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:43.774714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:43.774957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:43.775230 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:43.787367 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:02:43.793068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:43.794760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:43.796208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 624/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Sep 13 14:02:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:43.800989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f8a78-8d62-4302-825e-b29122ad4d3b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:43.852984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:43.914369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f8a78-8d62-4302-825e-b29122ad4d3b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:43.954278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:43.954958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:43.955256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:43.955563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:43.956281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:43.956623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:43.958006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:43.972187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 612/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Sep 13 14:02:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:43.973444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:44.071926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:44.090856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08348696-42a9-4f23-adb8-584b9b608df4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:44.131188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:44.131977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:44.132255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:44.132526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:44.145126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:44.146589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:44.164412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:44.170929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:44.172059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:44.172059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:44.172059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:44.172059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 625/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:44.172316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:44.290629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffd4d6-5335-45fb-8785-9e3130fd7d62 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:44.302764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:44.331990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:44.332529 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:44.332810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:44.333081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:44.333718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:44.334070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:44.335408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:44.348504 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 613/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:44.355519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:44.470822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f0bca-2d98-41bb-b62f-af8aef2fcf40 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:44.513754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:44.514417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:44.514693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:44.515081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:44.518049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:44.526922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:44.528911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:02:44.542537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:44.543696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:44.544718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 626/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 14:02:44.552077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:44.673957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cbfe3-65dc-4b61-8b65-1527fd5b2419 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:44.714740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:44.715571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:44.715882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:44.716230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:44.717084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:44.717557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:44.719501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:44.742109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 614/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 13 14:02:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:44.749127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:44.870756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9eae7-fb16-4d50-aa8f-ebc924d5ec89 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:44.908877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:44.909516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:44.909900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:44.910095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:44.981370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:44.989182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:44.990941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:45.100430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 627/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:02:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:02:45.107518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:45.137421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:45.229612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5dec2-5f07-48cf-82a8-15ba71f16d77 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:45.265907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:45.266469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:45.266746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:45.267017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:45.278796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:45.280750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:45.299620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:45.301278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:45.302355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:45.302355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:45.302355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:45.302355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 615/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:45.307264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:45.422445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:45.429435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0416c4-a860-4679-b961-274059a9e588 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:45.466072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:45.466683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:45.466959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:45.467266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:45.467939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:45.468240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:45.469800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:45.482620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 628/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:45.488351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:45.579850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:45.607842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d66308-5f37-4c3c-a081-b8214c3ed05f None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:45.649654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:45.650410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:45.650697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:45.650973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:45.658512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:45.660021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:02:45.671911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:45.673097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:45.674419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 616/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:45.679067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:45.794238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c8887-bf9e-4248-87ad-9a4adc1b9ba5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:45.834334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb07af1-8cd4-41b0-9de3-a3dfa01487b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:45.834987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb07af1-8cd4-41b0-9de3-a3dfa01487b1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:45.835305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb07af1-8cd4-41b0-9de3-a3dfa01487b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:45.835537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb07af1-8cd4-41b0-9de3-a3dfa01487b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:45.836619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb07af1-8cd4-41b0-9de3-a3dfa01487b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:45.837006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb07af1-8cd4-41b0-9de3-a3dfa01487b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:45.839343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb07af1-8cd4-41b0-9de3-a3dfa01487b1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:45.853087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 629/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:45.859613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:45.974699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c9a7b-1d67-4511-9ea6-05b4a73f8a9d None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:46.013409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f350622-5071-4276-a001-987c0ee56193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:46.014445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350622-5071-4276-a001-987c0ee56193 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:46.014740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350622-5071-4276-a001-987c0ee56193 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:46.015076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350622-5071-4276-a001-987c0ee56193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:46.015794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f350622-5071-4276-a001-987c0ee56193 None admin] RBAC: Adding query filter params (role.id=adad88d8a75747a381a2b923728d27f8, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=d09c0b02c24f4c5db5ed35fdee2589b5) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:46.016146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f350622-5071-4276-a001-987c0ee56193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:46.017624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f350622-5071-4276-a001-987c0ee56193 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:46.024377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 617/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:02:45 2022] GET /identity/v3/role_assignments?user.id=d09c0b02c24f4c5db5ed35fdee2589b5&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=adad88d8a75747a381a2b923728d27f8 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:48.379371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb81060-2cb5-435f-acf8-d0ff219547ed None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:48.379905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb81060-2cb5-435f-acf8-d0ff219547ed None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:48.379905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb81060-2cb5-435f-acf8-d0ff219547ed None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:48.381255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 630/1247] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:02:48.387080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6d6ff-6445-4165-8ca4-b7ce98e6139b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:48.387506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6d6ff-6445-4165-8ca4-b7ce98e6139b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:48.387616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6d6ff-6445-4165-8ca4-b7ce98e6139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:48.394265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:48.408382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d2f6d6ff-6445-4165-8ca4-b7ce98e6139b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:02:48.409654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d2f6d6ff-6445-4165-8ca4-b7ce98e6139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:02:48.483250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 618/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:02:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:02:48.774895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb81060-2cb5-435f-acf8-d0ff219547ed None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:48.796044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:48.895558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb81060-2cb5-435f-acf8-d0ff219547ed None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:48.934678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:48.935333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:48.935612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:48.935889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:48.947883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:02:48.953330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:48.954486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:48.955339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 631/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:48.960678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6d6ff-6445-4165-8ca4-b7ce98e6139b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:49.063303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6d6ff-6445-4165-8ca4-b7ce98e6139b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:49.094003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:49.099884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:49.100506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:49.100788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:49.101060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:49.101761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:49.102061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:49.103471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:49.113356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 619/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:02:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:49.119379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:49.209657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:49.235210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd652c-a36d-4cfe-bf61-6224779c83fc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:49.269258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:49.269955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:49.270236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:49.270511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:49.281559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:49.283025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:49.298508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:49.299547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:49.300453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:49.300453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:49.300453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:49.300453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 632/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:49.305482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:49.410366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bcab5-a2c9-4d34-bac5-688acfda56cc None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:49.444305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:49.444911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:49.445187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:49.445455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:49.446128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:49.446424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:49.447801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:49.460429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 620/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:49.466968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:49.567288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9108-c94d-49e6-afff-d45f6ee3fb6e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:49.591766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:49.599605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:49.600243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:49.600518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:49.600788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:49.608473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:49.609926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:02:49.621154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:49.622331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:49.623661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 633/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:49.628082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:49.682060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:49.724589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252043a-75d6-48f0-a997-51e7d0a5bbeb None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:49.756941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:49.757539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:49.757836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:49.758106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:49.758821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:49.759122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:49.761156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:49.778766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 621/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 13 14:02:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:49.784770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:49.882047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd041c-8ff3-4d4e-a376-d43a9947a7cf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:49.914591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:49.915175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:49.915452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:49.915721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:49.968467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:49.980774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:49.982252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:50.077319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 634/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:02:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:02:50.083080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:50.186645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0619f-ecf0-4800-b798-f56b973e821c None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:50.201788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:50.222714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:50.223276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:50.223554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:50.223829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:50.233669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:50.235122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:50.250083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:50.251220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:50.252506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:50.252506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:50.252506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:50.252506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 622/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:50.256914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:50.341717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:50.353601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f3c8d-33ab-4757-acd4-1e451e281ca5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:50.387490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:50.388088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:50.388371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:50.388647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:50.389281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:50.389585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:50.390991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:50.402260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 635/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:50.408073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:50.504798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186dd42-6bb8-4395-b80b-cb8b110e9b1b None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:50.537946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:50.538570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:50.538850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:50.539121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:50.546291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:50.547714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:02:50.559476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:50.560582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:50.561961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 623/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:50.566429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:50.666320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ef3c9-6e8a-4b88-927e-1de5cd809328 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:50.698471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee087f-8147-4e5a-9097-56b7f09f74d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:50.699061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee087f-8147-4e5a-9097-56b7f09f74d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:50.699338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee087f-8147-4e5a-9097-56b7f09f74d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:50.699627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee087f-8147-4e5a-9097-56b7f09f74d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:50.700280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee087f-8147-4e5a-9097-56b7f09f74d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:50.700580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee087f-8147-4e5a-9097-56b7f09f74d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:50.702017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee087f-8147-4e5a-9097-56b7f09f74d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:50.713992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 636/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:50.720099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:50.746669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:50.820252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f131bf1-e1fc-4818-9338-c266b9e25ec2 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:50.853046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96c7eb-7995-463f-81ba-1c7cb9424f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:50.853635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96c7eb-7995-463f-81ba-1c7cb9424f4d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:50.854022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96c7eb-7995-463f-81ba-1c7cb9424f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:50.855016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96c7eb-7995-463f-81ba-1c7cb9424f4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:50.855849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96c7eb-7995-463f-81ba-1c7cb9424f4d None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=d09c0b02c24f4c5db5ed35fdee2589b5) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:50.856161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96c7eb-7995-463f-81ba-1c7cb9424f4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:50.857576 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc96c7eb-7995-463f-81ba-1c7cb9424f4d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:50.862825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 624/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:02:50 2022] GET /identity/v3/role_assignments?user.id=d09c0b02c24f4c5db5ed35fdee2589b5&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=96af379f1477489b8542698d3c012f1a => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:52.994085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b4ee-42e3-4553-9e55-d3e4d36e1216 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:52.995089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b4ee-42e3-4553-9e55-d3e4d36e1216 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:52.995179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b4ee-42e3-4553-9e55-d3e4d36e1216 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:52.996733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 637/1261] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:02:53.001748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf31b08-83fc-4615-876f-c3c8b5075065 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:53.002120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf31b08-83fc-4615-876f-c3c8b5075065 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:53.002318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf31b08-83fc-4615-876f-c3c8b5075065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:53.021559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-daf31b08-83fc-4615-876f-c3c8b5075065 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:02:53.022784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-daf31b08-83fc-4615-876f-c3c8b5075065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:02:53.086542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 625/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:02:53.365029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648b4ee-42e3-4553-9e55-d3e4d36e1216 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:53.376505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:53.466616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648b4ee-42e3-4553-9e55-d3e4d36e1216 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:53.500390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:53.501007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:53.501289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:53.501562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:53.511726 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:02:53.516545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:53.517691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:53.518724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 638/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:53.523655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf31b08-83fc-4615-876f-c3c8b5075065 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:53.542476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:53.618781 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf31b08-83fc-4615-876f-c3c8b5075065 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:53.651222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:53.651816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:53.652086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:53.652352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:53.653003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:53.653301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:53.654916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:53.662542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 626/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:53.668970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:53.734096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:53.767531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b48578-88c1-490f-8e9f-9d49dd974901 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:53.800532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:53.801144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:53.801420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:53.801687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:53.811870 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:53.813290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:53.826993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:53.828108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:53.829026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:53.829026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:53.829026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:53.829026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 639/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:53.833746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:53.911891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:53.930741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758693a8-4f84-43cd-9c33-26ccfadb6489 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:53.961830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:53.962416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:53.962695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:53.962967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:53.964133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:53.964431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:53.965789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:53.976862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 627/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:53.983259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:54.079303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf131ba-7f20-4ead-9a1b-53354a4e0c0a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:54.098396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:54.112627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:54.113244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:54.113531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:54.113834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:54.121292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:54.122755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:02:54.133674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:54.134809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:54.136848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 640/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:54.141092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:54.244164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c356b4-4d9b-4028-9149-01080aece422 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:54.277574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:54.278200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:54.278482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:54.278755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:54.279837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:54.280146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:54.281567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:54.289960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:54.298908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 628/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 13 14:02:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:54.304707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:54.401798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765d79-dd1a-4087-9c4b-f910beef9ec8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:54.434594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:54.435206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:54.435484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:54.435756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:54.445823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:54.447253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:54.461608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:54.462562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:54.463771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:54.463771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:54.463771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:54.463771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 641/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:54.468384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:54.565215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768dc2-5a24-4a1e-9f6d-38acc1dbac31 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:54.598009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:54.598629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:54.598903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:54.599168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:54.599824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:54.600122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:54.601464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:54.612145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 629/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:54.617877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:54.649215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:54.716888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb834f-a66e-4232-88e0-5ed8f97dc50b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:54.749865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:54.750475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:54.750750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:54.751019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:54.757388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:54.758855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:02:54.770696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:54.771614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:54.772589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 642/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:54.777647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:54.835081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:54.876076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd439786-0db4-474e-9e1e-c11a3595d8ac None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:54.907742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba97f20-7529-43bf-a569-32821de4dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:54.908327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba97f20-7529-43bf-a569-32821de4dec5 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:54.908603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba97f20-7529-43bf-a569-32821de4dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:54.908869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba97f20-7529-43bf-a569-32821de4dec5 None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:54.909546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba97f20-7529-43bf-a569-32821de4dec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:54.909877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba97f20-7529-43bf-a569-32821de4dec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:54.911271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba97f20-7529-43bf-a569-32821de4dec5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:54.923484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 630/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:54.933231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:55.020584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:55.029340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29dd3c-1885-45ba-b265-aa109d79cec4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:55.061702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcafeaf-c3e0-4a50-81d4-6105706a6c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:55.062362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcafeaf-c3e0-4a50-81d4-6105706a6c73 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:55.062635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcafeaf-c3e0-4a50-81d4-6105706a6c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:55.062900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcafeaf-c3e0-4a50-81d4-6105706a6c73 None admin] PATH_INFO: `/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/d09c0b02c24f4c5db5ed35fdee2589b5/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:55.064751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcafeaf-c3e0-4a50-81d4-6105706a6c73 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ba472641f4e46b3884d884882bdaf, user_id=d09c0b02c24f4c5db5ed35fdee2589b5, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:55.066206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcafeaf-c3e0-4a50-81d4-6105706a6c73 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:55.078972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 643/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Sep 13 14:02:54 2022] PUT /identity/v3/projects/5e8ba472641f4e46b3884d884882bdaf/users/d09c0b02c24f4c5db5ed35fdee2589b5/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:02:57.196694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de3eb9-7f47-4fec-a0f3-00d407bc97ac None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:57.197274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de3eb9-7f47-4fec-a0f3-00d407bc97ac None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:57.197274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de3eb9-7f47-4fec-a0f3-00d407bc97ac None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:57.198867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 631/1274] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:02:57.205066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e0c7-9637-4fd3-ac3a-88b18785c909 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:57.205511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e0c7-9637-4fd3-ac3a-88b18785c909 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:57.206008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e0c7-9637-4fd3-ac3a-88b18785c909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:57.226248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5ae4e0c7-9637-4fd3-ac3a-88b18785c909 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:02:57.227423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5ae4e0c7-9637-4fd3-ac3a-88b18785c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:02:57.292179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 644/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:02:57.561517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de3eb9-7f47-4fec-a0f3-00d407bc97ac None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:57.660754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de3eb9-7f47-4fec-a0f3-00d407bc97ac None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:57.677650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:57.694130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:57.694746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:57.695022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:57.695290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:57.705194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 14:02:57.709980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:57.711156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:57.712038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 632/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Sep 13 14:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:57.717864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e0c7-9637-4fd3-ac3a-88b18785c909 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:57.795804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:57.814573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e0c7-9637-4fd3-ac3a-88b18785c909 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:57.848053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:57.848650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:57.848927 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:57.849196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:57.849942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:57.850245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:57.851598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:57.859894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 645/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Sep 13 14:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:57.866175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:57.964630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4a7ea-27ca-4259-8c93-2f4306ebbdc7 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:57.996880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:57.997487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:57.997790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:57.998068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:58.008026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:58.009460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:58.024374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:58.025556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:58.026789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:58.026789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:58.026789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:58.026789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 633/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:58.031702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:58.130789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc32154-0ea4-4ac0-84a6-6cf3665901aa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:58.166969 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:58.167602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:58.167954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:58.168220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:58.168868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:58.169174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:58.170677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:58.183706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 646/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:58.190238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:58.226000 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:58.294655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d658c94-4bfd-4199-9cf6-4ed328793c65 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:58.326376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:58.326991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:58.327268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:58.327560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:58.334636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:58.336062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 14:02:58.347514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:58.348627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:02:58.349972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 634/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:58.355310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:58.359072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:58.451430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05af2f-8a1b-446f-852a-e5e35877eeea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:58.484848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:58.485436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:58.485713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:58.486011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:58.486721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:58.487021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:58.488430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:58.504878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 647/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Sep 13 14:02:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:58.511087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:58.599970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:58.609883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7def2e-aa86-4603-ba50-a9eeeee8dda3 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:58.642991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:58.643584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:58.643860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:58.644130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:58.708926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:58.710336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:58.799523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:58.809942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 635/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Sep 13 14:02:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 14:02:58.815700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:58.876108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:58.913531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511d5fd-46a6-481a-b858-760167be8d48 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:58.946141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:58.946747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:58.947019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:58.947294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:58.957468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:58.958980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 14:02:58.973172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:58.974368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 14:02:58.975302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 14:02:58.975302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 14:02:58.975302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:58.975302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 648/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:58.980157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:59.076945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1644b9-589c-44f3-822f-6b8d0c8a4d46 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:59.109498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:59.110115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:59.110396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:59.110667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:59.111297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:59.111599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:59.112944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:59.124314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 636/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Sep 13 14:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:59.129831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:59.227567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba6579-2f7f-4876-bd50-01c40d9560e2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:59.240408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:59.261472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:59.262113 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:59.262395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:59.262668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:59.269068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:59.270526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 14:02:59.282028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:02:59.283064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 13 14:02:59.283986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 649/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Sep 13 14:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:02:59.288877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:59.337339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:02:59.386023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bb442-9128-45b7-809e-c7e6040a9a68 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:59.418374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47155a-7e93-4cd8-a835-b866ecd06a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:59.418966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47155a-7e93-4cd8-a835-b866ecd06a5e None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:59.419244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47155a-7e93-4cd8-a835-b866ecd06a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:59.419517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47155a-7e93-4cd8-a835-b866ecd06a5e None admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:59.420254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47155a-7e93-4cd8-a835-b866ecd06a5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:59.420563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47155a-7e93-4cd8-a835-b866ecd06a5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:59.422004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47155a-7e93-4cd8-a835-b866ecd06a5e None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:59.434387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 637/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Sep 13 14:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:02:59.440351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:02:59.540725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b799c-0e2c-487a-963f-9f7114de6502 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:02:59.574973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbb848-90c8-46b0-aef2-79e411d6a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:02:59.575564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbb848-90c8-46b0-aef2-79e411d6a0cc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:02:59.575846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbb848-90c8-46b0-aef2-79e411d6a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:02:59.576121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbb848-90c8-46b0-aef2-79e411d6a0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:02:59.576836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbb848-90c8-46b0-aef2-79e411d6a0cc None admin] RBAC: Adding query filter params (role.id=96af379f1477489b8542698d3c012f1a, scope.project.id=5e8ba472641f4e46b3884d884882bdaf, user.id=d09c0b02c24f4c5db5ed35fdee2589b5) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:02:59.577144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbb848-90c8-46b0-aef2-79e411d6a0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:02:59.578600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbb848-90c8-46b0-aef2-79e411d6a0cc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:02:59.584171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 650/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Sep 13 14:02:59 2022] GET /identity/v3/role_assignments?user.id=d09c0b02c24f4c5db5ed35fdee2589b5&scope.project.id=5e8ba472641f4e46b3884d884882bdaf&role.id=96af379f1477489b8542698d3c012f1a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:03:01.706141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd0b5e-d210-44ce-99fe-3704a0eb7904 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:01.706683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd0b5e-d210-44ce-99fe-3704a0eb7904 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:01.706683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd0b5e-d210-44ce-99fe-3704a0eb7904 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:01.708005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 638/1288] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:03:01.713317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1fe3c-413c-4bd4-a013-1dc5960e0877 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:01.713879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1fe3c-413c-4bd4-a013-1dc5960e0877 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:01.714241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1fe3c-413c-4bd4-a013-1dc5960e0877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:01.732530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-73b1fe3c-413c-4bd4-a013-1dc5960e0877 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:03:01.733698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73b1fe3c-413c-4bd4-a013-1dc5960e0877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:03:01.762471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:01.798637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 651/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:03:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:03:02.067027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd0b5e-d210-44ce-99fe-3704a0eb7904 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:02.169817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd0b5e-d210-44ce-99fe-3704a0eb7904 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:02.181900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:02.207589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:02.208259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:02.208580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:02.209010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:02.209470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:02.210994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:03:02.221549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:03:02.222710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:02.223600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 639/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:03:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:03:02.229922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1fe3c-413c-4bd4-a013-1dc5960e0877 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:02.326204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1fe3c-413c-4bd4-a013-1dc5960e0877 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:02.358754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666ab61-e22b-4012-bc92-e1e3311bca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:02.359345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ab61-e22b-4012-bc92-e1e3311bca59 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:02.359623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ab61-e22b-4012-bc92-e1e3311bca59 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:02.359894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666ab61-e22b-4012-bc92-e1e3311bca59 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:02.360521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666ab61-e22b-4012-bc92-e1e3311bca59 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:03:02.360850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666ab61-e22b-4012-bc92-e1e3311bca59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:02.362240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666ab61-e22b-4012-bc92-e1e3311bca59 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:02.366871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 652/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 13 14:03:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:03:02.372817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:02.469878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81698f-dddf-4bf0-8c5c-dd1d500a71d4 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:02.501633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b6ef7-24a4-474d-9504-c37e7d8aa67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:02.502243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b6ef7-24a4-474d-9504-c37e7d8aa67d None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:02.502520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b6ef7-24a4-474d-9504-c37e7d8aa67d None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:02.502788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b6ef7-24a4-474d-9504-c37e7d8aa67d None admin] PATH_INFO: `/v3/services` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:02.503400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b6ef7-24a4-474d-9504-c37e7d8aa67d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:03:02.503700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b6ef7-24a4-474d-9504-c37e7d8aa67d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:02.505033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b6ef7-24a4-474d-9504-c37e7d8aa67d None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:02.509328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 640/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 13 14:03:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:03:04.635213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618e657-be1a-4f66-911b-8469b51395d7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:04.635732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618e657-be1a-4f66-911b-8469b51395d7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:04.635917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618e657-be1a-4f66-911b-8469b51395d7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:04.637258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 653/1293] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:03:04.642625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69e5d-0fe5-419f-b17f-51fb1d229031 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:04.643067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69e5d-0fe5-419f-b17f-51fb1d229031 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:04.643163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c69e5d-0fe5-419f-b17f-51fb1d229031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:04.661156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f4c69e5d-0fe5-419f-b17f-51fb1d229031 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:03:04.662372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f4c69e5d-0fe5-419f-b17f-51fb1d229031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:03:04.691631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:04.727297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 641/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:03:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:03:04.995539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618e657-be1a-4f66-911b-8469b51395d7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:05.030752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:05.100465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618e657-be1a-4f66-911b-8469b51395d7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:05.132757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c140c45-4b4e-4b12-ab45-be03a121e323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:05.133346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c140c45-4b4e-4b12-ab45-be03a121e323 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:05.133622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c140c45-4b4e-4b12-ab45-be03a121e323 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:05.133928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c140c45-4b4e-4b12-ab45-be03a121e323 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:05.134576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c140c45-4b4e-4b12-ab45-be03a121e323 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:05.136525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c140c45-4b4e-4b12-ab45-be03a121e323 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:05.150891 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 654/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Sep 13 14:03:04 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:03:07.285455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86b265-5a31-41e5-aa8c-72c901d41007 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:07.286032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86b265-5a31-41e5-aa8c-72c901d41007 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:07.286032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86b265-5a31-41e5-aa8c-72c901d41007 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:07.287351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 642/1296] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:03:07.292740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c7d0-6f14-42a9-a5ef-75793ee3e624 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:07.293122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c7d0-6f14-42a9-a5ef-75793ee3e624 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:07.293281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843c7d0-6f14-42a9-a5ef-75793ee3e624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:07.311533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2843c7d0-6f14-42a9-a5ef-75793ee3e624 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:03:07.312996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2843c7d0-6f14-42a9-a5ef-75793ee3e624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:03:07.391280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 655/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:03:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:03:07.658090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86b265-5a31-41e5-aa8c-72c901d41007 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:07.761023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86b265-5a31-41e5-aa8c-72c901d41007 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:07.796811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:07.797465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:07.797792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:07.798089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:07.798724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:07.800224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:03:07.810660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:03:07.811868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:07.813054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 643/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:03:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:03:07.817845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843c7d0-6f14-42a9-a5ef-75793ee3e624 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:07.837070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:07.925149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843c7d0-6f14-42a9-a5ef-75793ee3e624 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:07.962746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:07.963881 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:07.964162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:07.964434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:07.965053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:03:07.965354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:07.966751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:07.971789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 656/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 13 14:03:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:03:07.977776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:08.008435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:08.075333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994db08-9854-4953-8831-c2fcc32e3597 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:08.108374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6889ba-ac10-4dc2-8359-8dbef804ee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:08.108955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6889ba-ac10-4dc2-8359-8dbef804ee03 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:08.109229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6889ba-ac10-4dc2-8359-8dbef804ee03 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:08.109495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6889ba-ac10-4dc2-8359-8dbef804ee03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:08.110207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6889ba-ac10-4dc2-8359-8dbef804ee03 None admin] RBAC: Adding query filter params (interface=public, service_id=253afd27b1bf43d9b0e2f408f6f70ff1, region_id=RegionOne) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:03:08.110509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6889ba-ac10-4dc2-8359-8dbef804ee03 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:08.111846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6889ba-ac10-4dc2-8359-8dbef804ee03 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:08.116613 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 644/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Sep 13 14:03:07 2022] GET /identity/v3/endpoints?service_id=253afd27b1bf43d9b0e2f408f6f70ff1&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:03:08.122264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:08.201065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:08.219463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3d1ea-5593-492a-b31e-f8889c89869a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:08.252293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46992e8-116b-4140-8d55-0e2056d45f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:08.252915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46992e8-116b-4140-8d55-0e2056d45f1f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:08.253195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46992e8-116b-4140-8d55-0e2056d45f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:08.253471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46992e8-116b-4140-8d55-0e2056d45f1f None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:08.254094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46992e8-116b-4140-8d55-0e2056d45f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:08.255450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46992e8-116b-4140-8d55-0e2056d45f1f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:08.262610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 657/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Sep 13 14:03:08 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:03:10.395055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754faf9-ab5c-4f37-858d-2cfe85085dd8 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:10.395596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754faf9-ab5c-4f37-858d-2cfe85085dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:10.395596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754faf9-ab5c-4f37-858d-2cfe85085dd8 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:10.396905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 645/1302] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:03:10.401997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309cbf-8aac-4302-b445-8c276151c57b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:10.402388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309cbf-8aac-4302-b445-8c276151c57b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:10.402532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25309cbf-8aac-4302-b445-8c276151c57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:10.420279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-25309cbf-8aac-4302-b445-8c276151c57b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:03:10.421444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-25309cbf-8aac-4302-b445-8c276151c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:03:10.484905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 658/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:03:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 14:03:10.750545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754faf9-ab5c-4f37-858d-2cfe85085dd8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:10.843324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:10.850216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754faf9-ab5c-4f37-858d-2cfe85085dd8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:10.883590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:10.884209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:10.884490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:10.885386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:10.885989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:10.887324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 14:03:10.895894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 14:03:10.897291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application with Lock( Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self._enter() Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application return self.creator() Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application created_value = creator( Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: ERROR keystone.server.flask.application Sep 13 14:03:10.898497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 646/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Sep 13 14:03:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 14:03:10.903014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25309cbf-8aac-4302-b445-8c276151c57b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:10.982929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:11.006577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25309cbf-8aac-4302-b445-8c276151c57b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:11.040033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b18c3-1f08-48f1-a0c3-11aedd493c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:11.040620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b18c3-1f08-48f1-a0c3-11aedd493c3f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:11.040897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b18c3-1f08-48f1-a0c3-11aedd493c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:11.041163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b18c3-1f08-48f1-a0c3-11aedd493c3f None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:11.041802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b18c3-1f08-48f1-a0c3-11aedd493c3f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:03:11.042103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b18c3-1f08-48f1-a0c3-11aedd493c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:11.043448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b18c3-1f08-48f1-a0c3-11aedd493c3f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:11.048383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 659/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Sep 13 14:03:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:03:11.054138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:03:11.151016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c32b-d283-4a31-96a2-c52fa35fc722 None admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:03:11.183102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6de88-14cd-4fb9-8c8d-8a677bb64de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:03:11.183689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6de88-14cd-4fb9-8c8d-8a677bb64de5 None admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:03:11.183962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6de88-14cd-4fb9-8c8d-8a677bb64de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:03:11.184230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6de88-14cd-4fb9-8c8d-8a677bb64de5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:03:11.184883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6de88-14cd-4fb9-8c8d-8a677bb64de5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:03:11.186262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6de88-14cd-4fb9-8c8d-8a677bb64de5 None admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:03:11.204897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 647/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:03:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:03:21.045589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=30 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:03:21.201180 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:15.288324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ff73e-5a32-4650-80e3-fcb9dfe1748f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:15.288920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ff73e-5a32-4650-80e3-fcb9dfe1748f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:15.288920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ff73e-5a32-4650-80e3-fcb9dfe1748f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:15.290603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 660/1307] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:15.295915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1d708-5bcc-4e30-923f-ecdc2d27960c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:15.296407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1d708-5bcc-4e30-923f-ecdc2d27960c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:15.296498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1d708-5bcc-4e30-923f-ecdc2d27960c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:15.464255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e8a1d708-5bcc-4e30-923f-ecdc2d27960c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:15.465483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e8a1d708-5bcc-4e30-923f-ecdc2d27960c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:15.704612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 648/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:16.012561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6396cd-de10-48c7-b65a-4556cdcbae9b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:16.013120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6396cd-de10-48c7-b65a-4556cdcbae9b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:16.013545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6396cd-de10-48c7-b65a-4556cdcbae9b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:16.015325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 661/1309] 10.222.0.20 () {54 vars in 965 bytes} [Tue Sep 13 14:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:16.022235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae700b-4328-4fdb-8fa4-2276cd55a381 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:16.022533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae700b-4328-4fdb-8fa4-2276cd55a381 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:16.022883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae700b-4328-4fdb-8fa4-2276cd55a381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:16.119262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-55ae700b-4328-4fdb-8fa4-2276cd55a381 None None] MFA Rules not processed for user `11a418a2a35146b59c433421fe507500`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:16.120565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-55ae700b-4328-4fdb-8fa4-2276cd55a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:16.144812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:16.225356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 649/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Sep 13 14:04:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:04:16.234934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6396cd-de10-48c7-b65a-4556cdcbae9b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:16.236670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ad6396cd-de10-48c7-b65a-4556cdcbae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:16.367532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6396cd-de10-48c7-b65a-4556cdcbae9b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:16.457663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb994e-9a9c-42a5-9ae0-1de5e2884465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:16.458703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb994e-9a9c-42a5-9ae0-1de5e2884465 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:16.459130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb994e-9a9c-42a5-9ae0-1de5e2884465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:16.459548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb994e-9a9c-42a5-9ae0-1de5e2884465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:16.461715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-67bb994e-9a9c-42a5-9ae0-1de5e2884465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:16.525958 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:16.593211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb994e-9a9c-42a5-9ae0-1de5e2884465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:16.594953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb994e-9a9c-42a5-9ae0-1de5e2884465 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:16.616658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 662/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:19.108840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7a45-ba89-4f13-95d6-6b85be87b3f3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:19.109378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7a45-ba89-4f13-95d6-6b85be87b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:19.109378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7a45-ba89-4f13-95d6-6b85be87b3f3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:19.110744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 650/1312] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:19.116112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68543faa-76c0-407d-a068-1fb9c226ab30 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:19.116531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68543faa-76c0-407d-a068-1fb9c226ab30 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:19.116664 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68543faa-76c0-407d-a068-1fb9c226ab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:19.187813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-68543faa-76c0-407d-a068-1fb9c226ab30 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:19.189016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68543faa-76c0-407d-a068-1fb9c226ab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:19.284852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 663/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:19.566170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f7a45-ba89-4f13-95d6-6b85be87b3f3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:19.567797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-da1f7a45-ba89-4f13-95d6-6b85be87b3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:19.674643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f7a45-ba89-4f13-95d6-6b85be87b3f3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:19.747954 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:19.750285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07202275-c166-4f4f-bdc0-6b0972e75e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:19.751098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07202275-c166-4f4f-bdc0-6b0972e75e37 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:19.751567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07202275-c166-4f4f-bdc0-6b0972e75e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:19.752027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07202275-c166-4f4f-bdc0-6b0972e75e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:19.753655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07202275-c166-4f4f-bdc0-6b0972e75e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:19.860347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07202275-c166-4f4f-bdc0-6b0972e75e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:19.861703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07202275-c166-4f4f-bdc0-6b0972e75e37 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:19.879782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 651/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:22.328519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddb21f-48dc-4001-a335-27f1f59b5b49 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:22.329104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddb21f-48dc-4001-a335-27f1f59b5b49 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:22.329104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddb21f-48dc-4001-a335-27f1f59b5b49 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:22.330425 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 664/1315] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:22.335506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d211de-7c28-4d6c-a746-fb1ed9d2972c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:22.336432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d211de-7c28-4d6c-a746-fb1ed9d2972c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:22.336578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d211de-7c28-4d6c-a746-fb1ed9d2972c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:22.401947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-11d211de-7c28-4d6c-a746-fb1ed9d2972c None None] MFA Rules not processed for user `dda66a8495034a4f94e940f3175970f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:22.403194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-11d211de-7c28-4d6c-a746-fb1ed9d2972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:22.455008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:22.500327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 652/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:22.786621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddb21f-48dc-4001-a335-27f1f59b5b49 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:22.788188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f9ddb21f-48dc-4001-a335-27f1f59b5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:22.798549 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:22.890535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddb21f-48dc-4001-a335-27f1f59b5b49 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:22.962492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee0378-c860-4a44-99f2-3b6de80108c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:22.963076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee0378-c860-4a44-99f2-3b6de80108c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:22.963352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee0378-c860-4a44-99f2-3b6de80108c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:22.963618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee0378-c860-4a44-99f2-3b6de80108c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:22.965606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-10ee0378-c860-4a44-99f2-3b6de80108c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:23.027209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:23.076331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee0378-c860-4a44-99f2-3b6de80108c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:23.077676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee0378-c860-4a44-99f2-3b6de80108c8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:23.095638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 665/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:25.774243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c6403-9885-4c20-84dc-01ca4fdbb76a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:25.774845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c6403-9885-4c20-84dc-01ca4fdbb76a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:25.774845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c6403-9885-4c20-84dc-01ca4fdbb76a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:25.776119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 653/1318] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:25.781568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebb8c6-6b3c-4f50-ab6f-6616a9707633 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:25.782029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebb8c6-6b3c-4f50-ab6f-6616a9707633 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:25.782173 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebb8c6-6b3c-4f50-ab6f-6616a9707633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:25.847844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f8ebb8c6-6b3c-4f50-ab6f-6616a9707633 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:25.849052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f8ebb8c6-6b3c-4f50-ab6f-6616a9707633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:25.945484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 666/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:26.230157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c6403-9885-4c20-84dc-01ca4fdbb76a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:26.231712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ae3c6403-9885-4c20-84dc-01ca4fdbb76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:26.334247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c6403-9885-4c20-84dc-01ca4fdbb76a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:26.407314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14173bc-efbd-4bf9-a0b9-b686902d3e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:26.407937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14173bc-efbd-4bf9-a0b9-b686902d3e18 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:26.408213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14173bc-efbd-4bf9-a0b9-b686902d3e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:26.408482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14173bc-efbd-4bf9-a0b9-b686902d3e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:26.410099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b14173bc-efbd-4bf9-a0b9-b686902d3e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:26.417277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:26.517689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14173bc-efbd-4bf9-a0b9-b686902d3e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:26.519070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14173bc-efbd-4bf9-a0b9-b686902d3e18 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:26.539674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 654/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:29.355736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6a93-a3a5-4ef7-a4d2-f499494cb6b3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:29.356278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6a93-a3a5-4ef7-a4d2-f499494cb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:29.356278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da6a93-a3a5-4ef7-a4d2-f499494cb6b3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:29.357829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 667/1321] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:29.363449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327799d-1210-41de-a25d-f1360fb022d9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:29.363977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327799d-1210-41de-a25d-f1360fb022d9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:29.364070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327799d-1210-41de-a25d-f1360fb022d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:29.438659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f327799d-1210-41de-a25d-f1360fb022d9 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:29.439971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f327799d-1210-41de-a25d-f1360fb022d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:29.503232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:29.546861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 655/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:04:29.850824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0ed50-b9c0-4655-b170-47ed235f40f3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:29.851258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0ed50-b9c0-4655-b170-47ed235f40f3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:29.851808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0ed50-b9c0-4655-b170-47ed235f40f3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:29.853343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 668/1323] 10.222.0.20 () {54 vars in 965 bytes} [Tue Sep 13 14:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:29.859388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281892-7c55-4368-b4e3-61ddfe9d948d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:29.859676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281892-7c55-4368-b4e3-61ddfe9d948d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:29.859950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281892-7c55-4368-b4e3-61ddfe9d948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:29.937129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b6281892-7c55-4368-b4e3-61ddfe9d948d None None] MFA Rules not processed for user `11a418a2a35146b59c433421fe507500`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:29.938373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b6281892-7c55-4368-b4e3-61ddfe9d948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:30.018014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 656/1324] 10.222.0.20 () {58 vars in 1062 bytes} [Tue Sep 13 14:04:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:04:30.027260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0ed50-b9c0-4655-b170-47ed235f40f3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:30.037905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1de0ed50-b9c0-4655-b170-47ed235f40f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:30.059776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:30.148027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0ed50-b9c0-4655-b170-47ed235f40f3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:30.222452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f4b68-78b7-416f-b2d1-b448217da311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:30.223048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f4b68-78b7-416f-b2d1-b448217da311 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:30.223322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f4b68-78b7-416f-b2d1-b448217da311 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:30.223586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f4b68-78b7-416f-b2d1-b448217da311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:30.225156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-942f4b68-78b7-416f-b2d1-b448217da311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:30.297118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:30.335585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f4b68-78b7-416f-b2d1-b448217da311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:30.337175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f4b68-78b7-416f-b2d1-b448217da311 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:30.356665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 669/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:32.808607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcf976-3bba-4ba1-95e4-27ef218ae6dd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:32.809133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcf976-3bba-4ba1-95e4-27ef218ae6dd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:32.809133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcf976-3bba-4ba1-95e4-27ef218ae6dd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:32.810459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 657/1326] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:32.815853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8625-cd63-4266-9b20-c4b685ee2344 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:32.816282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8625-cd63-4266-9b20-c4b685ee2344 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:32.816407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde8625-cd63-4266-9b20-c4b685ee2344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:32.880811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-acde8625-cd63-4266-9b20-c4b685ee2344 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:32.882022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-acde8625-cd63-4266-9b20-c4b685ee2344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:32.980412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 670/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:04:33.264140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcf976-3bba-4ba1-95e4-27ef218ae6dd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:33.265846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4fbcf976-3bba-4ba1-95e4-27ef218ae6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:33.328550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:33.372842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcf976-3bba-4ba1-95e4-27ef218ae6dd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:33.447411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a3ffe-8474-44a4-a1e9-c38167bd56d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:33.448017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a3ffe-8474-44a4-a1e9-c38167bd56d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:33.448296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a3ffe-8474-44a4-a1e9-c38167bd56d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:33.448567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a3ffe-8474-44a4-a1e9-c38167bd56d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:33.450209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cb2a3ffe-8474-44a4-a1e9-c38167bd56d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:33.564395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:33.566623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a3ffe-8474-44a4-a1e9-c38167bd56d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:33.568089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a3ffe-8474-44a4-a1e9-c38167bd56d1 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:33.590269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 658/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:35.801466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade118e6-e78d-4009-b15b-bc347edccb06 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:35.801939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade118e6-e78d-4009-b15b-bc347edccb06 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:35.802047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade118e6-e78d-4009-b15b-bc347edccb06 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:35.803327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 671/1329] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:35.810052 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc5441-a589-44a6-bcf5-dc750979d1b5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:35.810580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc5441-a589-44a6-bcf5-dc750979d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:35.810677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc5441-a589-44a6-bcf5-dc750979d1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:35.879521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6bcc5441-a589-44a6-bcf5-dc750979d1b5 None None] MFA Rules not processed for user `dda66a8495034a4f94e940f3175970f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:35.880844 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6bcc5441-a589-44a6-bcf5-dc750979d1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:35.981339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 659/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:36.273931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade118e6-e78d-4009-b15b-bc347edccb06 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:36.275470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ade118e6-e78d-4009-b15b-bc347edccb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:36.309296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:36.382855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade118e6-e78d-4009-b15b-bc347edccb06 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:36.460747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3822006-cabe-4b13-9215-7144c5d1b01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:36.461382 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3822006-cabe-4b13-9215-7144c5d1b01c service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:36.461794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3822006-cabe-4b13-9215-7144c5d1b01c service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:36.462071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3822006-cabe-4b13-9215-7144c5d1b01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:36.463671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d3822006-cabe-4b13-9215-7144c5d1b01c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:36.552784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:36.589857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3822006-cabe-4b13-9215-7144c5d1b01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:36.591300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3822006-cabe-4b13-9215-7144c5d1b01c service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:36.611883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 672/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:38.965009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1b732-5054-4134-ba0c-8c8ea8c61e29 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:38.965557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1b732-5054-4134-ba0c-8c8ea8c61e29 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:38.965557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1b732-5054-4134-ba0c-8c8ea8c61e29 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:38.966926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 660/1332] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:38.972046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1536a6-2e01-4a92-8d7a-46f806ff21e0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:38.972474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1536a6-2e01-4a92-8d7a-46f806ff21e0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:38.972594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1536a6-2e01-4a92-8d7a-46f806ff21e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:39.039323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2f1536a6-2e01-4a92-8d7a-46f806ff21e0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:39.040515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f1536a6-2e01-4a92-8d7a-46f806ff21e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:39.134225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 673/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:39.412891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1b732-5054-4134-ba0c-8c8ea8c61e29 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:39.415206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-92d1b732-5054-4134-ba0c-8c8ea8c61e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:39.483656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:39.517433 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1b732-5054-4134-ba0c-8c8ea8c61e29 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:39.593411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea10809-6231-4aaa-a87b-22cbf13f6bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:39.594052 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea10809-6231-4aaa-a87b-22cbf13f6bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:39.594336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea10809-6231-4aaa-a87b-22cbf13f6bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:39.594609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea10809-6231-4aaa-a87b-22cbf13f6bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:39.596202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ea10809-6231-4aaa-a87b-22cbf13f6bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:39.704724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea10809-6231-4aaa-a87b-22cbf13f6bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:39.706195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea10809-6231-4aaa-a87b-22cbf13f6bbb service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:39.716891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:39.725334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 661/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:41.917075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e02bb-aafc-4722-9d55-a455ca62da84 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:41.917618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e02bb-aafc-4722-9d55-a455ca62da84 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:41.917618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e02bb-aafc-4722-9d55-a455ca62da84 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:41.919028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 674/1335] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:41.924656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddd3a0-95a1-4734-9d28-57d4fa8a392d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:41.925090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddd3a0-95a1-4734-9d28-57d4fa8a392d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:41.925217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddd3a0-95a1-4734-9d28-57d4fa8a392d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:41.990529 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dbddd3a0-95a1-4734-9d28-57d4fa8a392d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:41.992313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dbddd3a0-95a1-4734-9d28-57d4fa8a392d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:42.087930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 662/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:42.371732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e02bb-aafc-4722-9d55-a455ca62da84 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:42.373308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1c5e02bb-aafc-4722-9d55-a455ca62da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:42.414055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:42.477042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e02bb-aafc-4722-9d55-a455ca62da84 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:42.553160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97a194-8e79-4c2b-8ab1-0dd3aaf7873a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:42.553783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97a194-8e79-4c2b-8ab1-0dd3aaf7873a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:42.554060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97a194-8e79-4c2b-8ab1-0dd3aaf7873a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:42.554325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97a194-8e79-4c2b-8ab1-0dd3aaf7873a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:42.555893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5c97a194-8e79-4c2b-8ab1-0dd3aaf7873a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:42.638459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:42.664646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97a194-8e79-4c2b-8ab1-0dd3aaf7873a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:42.666115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97a194-8e79-4c2b-8ab1-0dd3aaf7873a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:42.684124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 675/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:45.156542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cee1a-360e-49dc-b048-931a96b3bf01 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:45.157190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cee1a-360e-49dc-b048-931a96b3bf01 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:45.157190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cee1a-360e-49dc-b048-931a96b3bf01 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:45.158453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 663/1338] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:45.164603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb259d-83e8-4f90-affa-8f71844f4f0d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:45.165103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb259d-83e8-4f90-affa-8f71844f4f0d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:45.165285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb259d-83e8-4f90-affa-8f71844f4f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:45.230988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4efb259d-83e8-4f90-affa-8f71844f4f0d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:45.232107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4efb259d-83e8-4f90-affa-8f71844f4f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:45.326949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 676/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:45.611377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cee1a-360e-49dc-b048-931a96b3bf01 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:45.612955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6e9cee1a-360e-49dc-b048-931a96b3bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:45.688007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:45.716533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cee1a-360e-49dc-b048-931a96b3bf01 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:45.791163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d788c9b-34fb-494e-ba6f-c4156a5bbc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:45.791680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788c9b-34fb-494e-ba6f-c4156a5bbc92 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:45.791958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788c9b-34fb-494e-ba6f-c4156a5bbc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:45.792228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788c9b-34fb-494e-ba6f-c4156a5bbc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:45.793850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6d788c9b-34fb-494e-ba6f-c4156a5bbc92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:45.901323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d788c9b-34fb-494e-ba6f-c4156a5bbc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:45.902780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d788c9b-34fb-494e-ba6f-c4156a5bbc92 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:45.920764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 664/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:49.959116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c0e2-73a6-4a5d-895c-719875704d38 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:49.959639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c0e2-73a6-4a5d-895c-719875704d38 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:49.959639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c0e2-73a6-4a5d-895c-719875704d38 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:49.961702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 677/1341] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:49.967112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af6bb8-b24b-4f51-96e2-19d655f9aa6a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:49.967540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af6bb8-b24b-4f51-96e2-19d655f9aa6a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:49.967674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af6bb8-b24b-4f51-96e2-19d655f9aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:49.973507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:50.035600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-89af6bb8-b24b-4f51-96e2-19d655f9aa6a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:50.036707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-89af6bb8-b24b-4f51-96e2-19d655f9aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:50.133394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 665/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:04:50.419111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7c0e2-73a6-4a5d-895c-719875704d38 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:50.420647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5be7c0e2-73a6-4a5d-895c-719875704d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:50.470261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:50.530627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7c0e2-73a6-4a5d-895c-719875704d38 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:50.606219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebfa6d-0dd2-4750-b159-d3b42a2d265b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:50.606823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebfa6d-0dd2-4750-b159-d3b42a2d265b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:50.607103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebfa6d-0dd2-4750-b159-d3b42a2d265b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:50.607375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebfa6d-0dd2-4750-b159-d3b42a2d265b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:50.608963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fcebfa6d-0dd2-4750-b159-d3b42a2d265b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:50.706207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:50.723285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebfa6d-0dd2-4750-b159-d3b42a2d265b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:50.724717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebfa6d-0dd2-4750-b159-d3b42a2d265b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:50.742286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 678/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:53.572616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ea263-54d1-4986-bee8-41c4b0bc8909 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:53.573200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ea263-54d1-4986-bee8-41c4b0bc8909 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:53.573200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ea263-54d1-4986-bee8-41c4b0bc8909 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:53.574534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 666/1344] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:53.579772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32f93-a2a4-462f-a27e-27899ef4e033 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:53.580209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32f93-a2a4-462f-a27e-27899ef4e033 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:53.580337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a32f93-a2a4-462f-a27e-27899ef4e033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:53.645141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-07a32f93-a2a4-462f-a27e-27899ef4e033 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:53.646357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-07a32f93-a2a4-462f-a27e-27899ef4e033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:53.741680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 679/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:04:54.023441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ea263-54d1-4986-bee8-41c4b0bc8909 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:54.024992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-199ea263-54d1-4986-bee8-41c4b0bc8909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:54.123591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:54.142840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ea263-54d1-4986-bee8-41c4b0bc8909 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:54.222504 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f63830-8530-4ef0-8be8-ae14031df7de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:54.223105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f63830-8530-4ef0-8be8-ae14031df7de service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:54.223383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f63830-8530-4ef0-8be8-ae14031df7de service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:54.223654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f63830-8530-4ef0-8be8-ae14031df7de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:54.225290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f4f63830-8530-4ef0-8be8-ae14031df7de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:54.334716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f63830-8530-4ef0-8be8-ae14031df7de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:54.336199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f63830-8530-4ef0-8be8-ae14031df7de service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:54.354407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 667/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:04:59.078839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954ce43-f3ef-40f4-a209-e1ea2f5a78e3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:59.079337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954ce43-f3ef-40f4-a209-e1ea2f5a78e3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:59.079431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954ce43-f3ef-40f4-a209-e1ea2f5a78e3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:59.080907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 680/1347] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:04:59.086812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4038-aaed-4eec-af9d-c5654aa17516 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:59.087246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4038-aaed-4eec-af9d-c5654aa17516 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:59.087353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4038-aaed-4eec-af9d-c5654aa17516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:59.103621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:59.164935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9ebc4038-aaed-4eec-af9d-c5654aa17516 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:04:59.166379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9ebc4038-aaed-4eec-af9d-c5654aa17516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:59.269771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 668/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:04:59.551871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954ce43-f3ef-40f4-a209-e1ea2f5a78e3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:04:59.553610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5954ce43-f3ef-40f4-a209-e1ea2f5a78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:59.611535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:59.657834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954ce43-f3ef-40f4-a209-e1ea2f5a78e3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:04:59.731648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07bdae-c2bf-4b5a-b999-22cd3ce2822e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:04:59.732249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07bdae-c2bf-4b5a-b999-22cd3ce2822e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:04:59.732530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07bdae-c2bf-4b5a-b999-22cd3ce2822e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:04:59.732807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07bdae-c2bf-4b5a-b999-22cd3ce2822e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:04:59.734439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cc07bdae-c2bf-4b5a-b999-22cd3ce2822e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:04:59.839919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:04:59.844108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07bdae-c2bf-4b5a-b999-22cd3ce2822e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:04:59.845528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07bdae-c2bf-4b5a-b999-22cd3ce2822e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:04:59.863351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 681/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:04.746469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273f06-6d33-438e-994d-5fdc191b8674 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:04.746928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273f06-6d33-438e-994d-5fdc191b8674 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:04.747044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42273f06-6d33-438e-994d-5fdc191b8674 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:04.748631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 669/1350] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:04.754367 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69667102-51e0-441a-8b80-f906a456728c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:04.754799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69667102-51e0-441a-8b80-f906a456728c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:04.754904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69667102-51e0-441a-8b80-f906a456728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:04.857075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-69667102-51e0-441a-8b80-f906a456728c None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:04.860575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-69667102-51e0-441a-8b80-f906a456728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:05.047420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 682/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:05:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:09.268108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=59 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:09.274299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcec34e-9545-455d-8bc1-553c4e90b683 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:09.274569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcec34e-9545-455d-8bc1-553c4e90b683 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:09.275457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcec34e-9545-455d-8bc1-553c4e90b683 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:09.277536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 670/1352] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:05:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:09.283020 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18c56-967b-4d62-88d0-7a7330b88e8a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:09.283462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18c56-967b-4d62-88d0-7a7330b88e8a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:09.283669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18c56-967b-4d62-88d0-7a7330b88e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:09.342502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:09.368564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-67f18c56-967b-4d62-88d0-7a7330b88e8a None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:09.369936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-67f18c56-967b-4d62-88d0-7a7330b88e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:09.481807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 683/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:10.781830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511ca3b-8a94-4a26-a504-e96080f2f3d4 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:10.782347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511ca3b-8a94-4a26-a504-e96080f2f3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:10.782467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511ca3b-8a94-4a26-a504-e96080f2f3d4 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:10.783695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 671/1354] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:10.789600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614a3ac-786f-43e6-aecd-5dc25e9f63ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:10.790400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614a3ac-786f-43e6-aecd-5dc25e9f63ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:10.790685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614a3ac-786f-43e6-aecd-5dc25e9f63ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:10.872202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0614a3ac-786f-43e6-aecd-5dc25e9f63ff None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:10.873409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0614a3ac-786f-43e6-aecd-5dc25e9f63ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:10.921593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:10.991088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 684/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:05:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:05:14.647875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4d0fb-5b1e-41f1-b40d-f7a52e6ebb41 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:14.648383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4d0fb-5b1e-41f1-b40d-f7a52e6ebb41 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:14.648383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4d0fb-5b1e-41f1-b40d-f7a52e6ebb41 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:14.649907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 672/1356] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:14.655377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da33dd-1a5d-4c29-ab09-82cfb7fcca44 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:14.655783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da33dd-1a5d-4c29-ab09-82cfb7fcca44 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:14.655963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da33dd-1a5d-4c29-ab09-82cfb7fcca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:14.725084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-99da33dd-1a5d-4c29-ab09-82cfb7fcca44 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:14.726340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-99da33dd-1a5d-4c29-ab09-82cfb7fcca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:14.831156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 685/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:15.133242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928e8f0-7548-468f-bff9-e5e817ecbf8a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:15.134053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928e8f0-7548-468f-bff9-e5e817ecbf8a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:15.134745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928e8f0-7548-468f-bff9-e5e817ecbf8a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:15.136496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 673/1358] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 13 14:05:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:15.144511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a552a-b48b-4288-80fd-ed8ae0ad0745 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:15.144873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a552a-b48b-4288-80fd-ed8ae0ad0745 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:15.145224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a552a-b48b-4288-80fd-ed8ae0ad0745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:15.163812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:15.232151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-722a552a-b48b-4288-80fd-ed8ae0ad0745 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:15.233494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-722a552a-b48b-4288-80fd-ed8ae0ad0745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:15.361006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 686/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 13 14:05:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:05:15.370313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928e8f0-7548-468f-bff9-e5e817ecbf8a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:15.371895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a928e8f0-7548-468f-bff9-e5e817ecbf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:15.532683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928e8f0-7548-468f-bff9-e5e817ecbf8a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:15.612455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:15.657765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a032e2-65f4-4507-8782-651ce457b6a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:15.658421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a032e2-65f4-4507-8782-651ce457b6a5 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:15.658697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a032e2-65f4-4507-8782-651ce457b6a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:15.658912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a032e2-65f4-4507-8782-651ce457b6a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:15.660680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-81a032e2-65f4-4507-8782-651ce457b6a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:15.759551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a032e2-65f4-4507-8782-651ce457b6a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:15.761524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a032e2-65f4-4507-8782-651ce457b6a5 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:15.782243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 674/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:18.201645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151559e-9530-47bc-a94f-bbb5479011be None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:18.202513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151559e-9530-47bc-a94f-bbb5479011be None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:18.202513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151559e-9530-47bc-a94f-bbb5479011be None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:18.203914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 687/1361] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:18.210428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1906c-c38e-4677-a939-f0dd179daec6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:18.210983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1906c-c38e-4677-a939-f0dd179daec6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:18.210983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1906c-c38e-4677-a939-f0dd179daec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:18.290703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e6d1906c-c38e-4677-a939-f0dd179daec6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:18.292073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6d1906c-c38e-4677-a939-f0dd179daec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:18.336513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:18.417787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 675/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:18.751723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151559e-9530-47bc-a94f-bbb5479011be None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:18.753342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a151559e-9530-47bc-a94f-bbb5479011be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:18.838163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:18.950718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151559e-9530-47bc-a94f-bbb5479011be None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:19.089039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234a407-5819-4c72-9b76-920662b1522a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:19.089609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234a407-5819-4c72-9b76-920662b1522a service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:19.089994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234a407-5819-4c72-9b76-920662b1522a service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:19.090234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234a407-5819-4c72-9b76-920662b1522a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:19.091858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0234a407-5819-4c72-9b76-920662b1522a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:19.117019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:19.194817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234a407-5819-4c72-9b76-920662b1522a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:19.196451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234a407-5819-4c72-9b76-920662b1522a service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:19.222680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 688/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:21.750696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50833b2-4e67-44b0-91ed-0ddbb7f87042 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:21.751220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50833b2-4e67-44b0-91ed-0ddbb7f87042 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:21.751303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50833b2-4e67-44b0-91ed-0ddbb7f87042 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:21.752752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 676/1364] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:21.758472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b99bf1-77e0-4794-8b6f-0b810f53defc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:21.758965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b99bf1-77e0-4794-8b6f-0b810f53defc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:21.759234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b99bf1-77e0-4794-8b6f-0b810f53defc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:21.831796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-37b99bf1-77e0-4794-8b6f-0b810f53defc None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:21.833084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37b99bf1-77e0-4794-8b6f-0b810f53defc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:21.933824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:21.944697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 689/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:22.239602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3be33-5654-4110-8d46-abd19ebc4e15 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:22.240265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3be33-5654-4110-8d46-abd19ebc4e15 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:22.240745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3be33-5654-4110-8d46-abd19ebc4e15 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:22.242434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 677/1366] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 13 14:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:22.248941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f529cd-182d-4f53-bfc3-d597a38a856b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:22.249224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f529cd-182d-4f53-bfc3-d597a38a856b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:22.249492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f529cd-182d-4f53-bfc3-d597a38a856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:22.332070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-39f529cd-182d-4f53-bfc3-d597a38a856b None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:22.333267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-39f529cd-182d-4f53-bfc3-d597a38a856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:22.457847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 690/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 13 14:05:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:22.466166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3be33-5654-4110-8d46-abd19ebc4e15 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:22.467674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-45a3be33-5654-4110-8d46-abd19ebc4e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:22.635024 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3be33-5654-4110-8d46-abd19ebc4e15 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:22.656505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:22.756447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d23b32-1859-414f-93b6-151763bf46e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:22.757333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d23b32-1859-414f-93b6-151763bf46e3 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:22.757604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d23b32-1859-414f-93b6-151763bf46e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:22.757815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d23b32-1859-414f-93b6-151763bf46e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:22.759383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d1d23b32-1859-414f-93b6-151763bf46e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:22.854540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d23b32-1859-414f-93b6-151763bf46e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:22.856218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d23b32-1859-414f-93b6-151763bf46e3 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:22.878046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 678/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:23.044384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613ffec-9828-4b62-8c54-9816ff38e8e8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:23.044978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613ffec-9828-4b62-8c54-9816ff38e8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:23.044978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613ffec-9828-4b62-8c54-9816ff38e8e8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:23.046522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 691/1369] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:23.054026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266f39-dfdf-4356-8deb-62b0aca41b93 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:23.054388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266f39-dfdf-4356-8deb-62b0aca41b93 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:23.054692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f266f39-dfdf-4356-8deb-62b0aca41b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:23.096184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:23.143942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3f266f39-dfdf-4356-8deb-62b0aca41b93 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:23.145321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3f266f39-dfdf-4356-8deb-62b0aca41b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:23.266722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 679/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:23.563213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019c0c-3778-4db2-8723-3e039c41a59f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:23.563660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019c0c-3778-4db2-8723-3e039c41a59f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:23.564032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23019c0c-3778-4db2-8723-3e039c41a59f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:23.565570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 692/1371] 10.222.0.20 () {54 vars in 964 bytes} [Tue Sep 13 14:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:23.571460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af275b2-f9fc-455f-b713-2edaee14b7c7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:23.571751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af275b2-f9fc-455f-b713-2edaee14b7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:23.572020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af275b2-f9fc-455f-b713-2edaee14b7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:23.655327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7af275b2-f9fc-455f-b713-2edaee14b7c7 None None] MFA Rules not processed for user `c95553acbf23423993a4d25e83ecee48`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:23.656788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7af275b2-f9fc-455f-b713-2edaee14b7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:23.709747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:23.744277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 680/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Sep 13 14:05:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:23.752086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23019c0c-3778-4db2-8723-3e039c41a59f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:23.753628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-23019c0c-3778-4db2-8723-3e039c41a59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:23.811829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:23.880352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23019c0c-3778-4db2-8723-3e039c41a59f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:23.964544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3d2b9-b0d2-4fbf-8af3-53b67b6d004f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:23.964544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3d2b9-b0d2-4fbf-8af3-53b67b6d004f service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:23.965522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3d2b9-b0d2-4fbf-8af3-53b67b6d004f service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:23.965522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3d2b9-b0d2-4fbf-8af3-53b67b6d004f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:23.967023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b8b3d2b9-b0d2-4fbf-8af3-53b67b6d004f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:24.072033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:24.089839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3d2b9-b0d2-4fbf-8af3-53b67b6d004f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:24.091674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3d2b9-b0d2-4fbf-8af3-53b67b6d004f service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:24.116568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 693/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:25.148598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938497f-8a64-4a39-9b80-43293bc5478a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:25.149104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938497f-8a64-4a39-9b80-43293bc5478a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:25.149195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938497f-8a64-4a39-9b80-43293bc5478a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:25.150586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 681/1374] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:25.156374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ac9ef-e398-45c6-b74a-fb3e28a20e63 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:25.156781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ac9ef-e398-45c6-b74a-fb3e28a20e63 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:25.156935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ac9ef-e398-45c6-b74a-fb3e28a20e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:25.224082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9f9ac9ef-e398-45c6-b74a-fb3e28a20e63 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:25.225980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9f9ac9ef-e398-45c6-b74a-fb3e28a20e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:25.326830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 694/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:25.617557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938497f-8a64-4a39-9b80-43293bc5478a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:25.618942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4938497f-8a64-4a39-9b80-43293bc5478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:25.776570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938497f-8a64-4a39-9b80-43293bc5478a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:25.837387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:25.908023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73596d21-4beb-46fa-a7c4-e2b482baf026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:25.908634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596d21-4beb-46fa-a7c4-e2b482baf026 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:25.908926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596d21-4beb-46fa-a7c4-e2b482baf026 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:25.909183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73596d21-4beb-46fa-a7c4-e2b482baf026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:25.910867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-73596d21-4beb-46fa-a7c4-e2b482baf026 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:26.012470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73596d21-4beb-46fa-a7c4-e2b482baf026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:26.014269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73596d21-4beb-46fa-a7c4-e2b482baf026 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:26.034004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 682/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:28.306579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fed04-225b-44bd-ae1e-08ae4e2612e4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:28.307137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fed04-225b-44bd-ae1e-08ae4e2612e4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:28.307137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fed04-225b-44bd-ae1e-08ae4e2612e4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:28.309810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 695/1377] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:28.316013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e6fc6-6d40-43af-a77b-221ef798a808 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:28.316460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e6fc6-6d40-43af-a77b-221ef798a808 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:28.316542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e6fc6-6d40-43af-a77b-221ef798a808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:28.387962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5a6e6fc6-6d40-43af-a77b-221ef798a808 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:28.389195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5a6e6fc6-6d40-43af-a77b-221ef798a808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:28.397178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:28.496687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 683/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:05:28.797014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fed04-225b-44bd-ae1e-08ae4e2612e4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:28.798750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca0fed04-225b-44bd-ae1e-08ae4e2612e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:28.862601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:28.951981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fed04-225b-44bd-ae1e-08ae4e2612e4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:29.072669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3d5e5-d4b8-4743-9054-efa427222165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:29.073421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3d5e5-d4b8-4743-9054-efa427222165 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:29.073702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3d5e5-d4b8-4743-9054-efa427222165 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:29.074003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3d5e5-d4b8-4743-9054-efa427222165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:29.075595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eef3d5e5-d4b8-4743-9054-efa427222165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:29.096791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:29.172513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3d5e5-d4b8-4743-9054-efa427222165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:29.174241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3d5e5-d4b8-4743-9054-efa427222165 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:29.193429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 696/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:31.425434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273cff8-0ec9-4d00-b3d9-2c1872be764f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:31.426008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273cff8-0ec9-4d00-b3d9-2c1872be764f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:31.426008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273cff8-0ec9-4d00-b3d9-2c1872be764f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:31.427684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 684/1380] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:31.432961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279ca2-eaec-4436-9a19-26232769b6a9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:31.433377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279ca2-eaec-4436-9a19-26232769b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:31.433546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279ca2-eaec-4436-9a19-26232769b6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:31.502348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f9279ca2-eaec-4436-9a19-26232769b6a9 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:31.503554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f9279ca2-eaec-4436-9a19-26232769b6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:31.603214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:31.610561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 697/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:31.911502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273cff8-0ec9-4d00-b3d9-2c1872be764f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:31.913092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9273cff8-0ec9-4d00-b3d9-2c1872be764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:32.063958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:32.082095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273cff8-0ec9-4d00-b3d9-2c1872be764f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:32.211561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1214f-3fe8-4fe5-9298-1967cba834b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:32.212164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1214f-3fe8-4fe5-9298-1967cba834b7 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:32.212444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1214f-3fe8-4fe5-9298-1967cba834b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:32.212712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1214f-3fe8-4fe5-9298-1967cba834b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:32.214344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0bb1214f-3fe8-4fe5-9298-1967cba834b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:32.357675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1214f-3fe8-4fe5-9298-1967cba834b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:32.359223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1214f-3fe8-4fe5-9298-1967cba834b7 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:32.374725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:32.385048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 685/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:35.110507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb396674-f523-476d-826f-ebbf5c1e6dd6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:35.111077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb396674-f523-476d-826f-ebbf5c1e6dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:35.111204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb396674-f523-476d-826f-ebbf5c1e6dd6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:35.113771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 698/1383] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:35.120582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b06a7-88ef-4313-8ed7-a77655453336 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:35.121058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b06a7-88ef-4313-8ed7-a77655453336 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:35.121289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b06a7-88ef-4313-8ed7-a77655453336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:35.203479 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-972b06a7-88ef-4313-8ed7-a77655453336 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:35.205049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-972b06a7-88ef-4313-8ed7-a77655453336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:35.319205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 686/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:35.682227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb396674-f523-476d-826f-ebbf5c1e6dd6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:35.684209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bb396674-f523-476d-826f-ebbf5c1e6dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:35.872758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb396674-f523-476d-826f-ebbf5c1e6dd6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:35.958830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:36.014148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca28b4-830e-42cf-ba34-160588edca16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:36.014852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca28b4-830e-42cf-ba34-160588edca16 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:36.015198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca28b4-830e-42cf-ba34-160588edca16 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:36.015536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca28b4-830e-42cf-ba34-160588edca16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:36.017360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-38ca28b4-830e-42cf-ba34-160588edca16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:36.123639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca28b4-830e-42cf-ba34-160588edca16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:36.125155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca28b4-830e-42cf-ba34-160588edca16 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:36.143841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 699/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:38.533905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07f094-4e80-4e85-ad4f-bd94fbe1a6f7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:38.534461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07f094-4e80-4e85-ad4f-bd94fbe1a6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:38.534753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07f094-4e80-4e85-ad4f-bd94fbe1a6f7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:38.536978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 687/1386] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:38.542924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8693-03c2-49c9-aae6-9d7f40d2dabf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:38.543345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8693-03c2-49c9-aae6-9d7f40d2dabf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:38.543521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8693-03c2-49c9-aae6-9d7f40d2dabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:38.613706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f48e8693-03c2-49c9-aae6-9d7f40d2dabf None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:38.615172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f48e8693-03c2-49c9-aae6-9d7f40d2dabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:38.654159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:38.719506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 700/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:39.013545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07f094-4e80-4e85-ad4f-bd94fbe1a6f7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:39.015194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1a07f094-4e80-4e85-ad4f-bd94fbe1a6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:39.093382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:39.164306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07f094-4e80-4e85-ad4f-bd94fbe1a6f7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:39.281514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50680b-6cd7-4861-b8b3-78f917b1fbb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:39.282062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50680b-6cd7-4861-b8b3-78f917b1fbb5 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:39.282352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50680b-6cd7-4861-b8b3-78f917b1fbb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:39.282627 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50680b-6cd7-4861-b8b3-78f917b1fbb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:39.284226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fb50680b-6cd7-4861-b8b3-78f917b1fbb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:39.322957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:39.375517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50680b-6cd7-4861-b8b3-78f917b1fbb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:39.377103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50680b-6cd7-4861-b8b3-78f917b1fbb5 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:39.394859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 688/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:39.638732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ab7a-5330-4bec-ab8a-933c998b7a07 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:39.639241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ab7a-5330-4bec-ab8a-933c998b7a07 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:39.639241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ab7a-5330-4bec-ab8a-933c998b7a07 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:39.640554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 701/1389] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:39.646675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117c56e-64f5-4a89-a4ee-cc5b13c26230 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:39.647253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117c56e-64f5-4a89-a4ee-cc5b13c26230 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:39.647253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117c56e-64f5-4a89-a4ee-cc5b13c26230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:39.715591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6117c56e-64f5-4a89-a4ee-cc5b13c26230 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:39.716808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6117c56e-64f5-4a89-a4ee-cc5b13c26230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:39.818363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 689/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:40.127449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0dba-5b15-465e-8cfd-61d2b4c5343d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:40.127725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0dba-5b15-465e-8cfd-61d2b4c5343d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:40.127993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0dba-5b15-465e-8cfd-61d2b4c5343d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:40.129328 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 702/1391] 10.222.0.20 () {54 vars in 966 bytes} [Tue Sep 13 14:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:40.135598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98b97f-fe9c-41e8-a13a-b8a711c70b13 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:40.136116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98b97f-fe9c-41e8-a13a-b8a711c70b13 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:40.136460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98b97f-fe9c-41e8-a13a-b8a711c70b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:40.152723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:40.216567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6d98b97f-fe9c-41e8-a13a-b8a711c70b13 None None] MFA Rules not processed for user `f7214e6addc646a899efb82283ededc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:40.218111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6d98b97f-fe9c-41e8-a13a-b8a711c70b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:40.294677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 690/1392] 10.222.0.20 () {58 vars in 1063 bytes} [Tue Sep 13 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:40.304700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d0dba-5b15-465e-8cfd-61d2b4c5343d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:40.306412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-189d0dba-5b15-465e-8cfd-61d2b4c5343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:40.412898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d0dba-5b15-465e-8cfd-61d2b4c5343d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:40.475899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:40.492274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd27ea6-9efc-4218-99e7-d30472e1be72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7214e6addc646a899efb82283ededc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:40.492877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27ea6-9efc-4218-99e7-d30472e1be72 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:40.493190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27ea6-9efc-4218-99e7-d30472e1be72 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:40.493459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd27ea6-9efc-4218-99e7-d30472e1be72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:40.495123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1fd27ea6-9efc-4218-99e7-d30472e1be72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:40.620347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd27ea6-9efc-4218-99e7-d30472e1be72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:40.621831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd27ea6-9efc-4218-99e7-d30472e1be72 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:40.640743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 703/1393] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:41.401518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d9dc7-d5cd-45cc-8367-4a16de25333c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:41.402056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d9dc7-d5cd-45cc-8367-4a16de25333c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:41.402336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d9dc7-d5cd-45cc-8367-4a16de25333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:41.485268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e21d9dc7-d5cd-45cc-8367-4a16de25333c None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:41.486550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e21d9dc7-d5cd-45cc-8367-4a16de25333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:41.548976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:41.581308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 691/1394] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:41.603666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a7244-aa7a-4bff-95c0-f7a9357ed2e7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:41.604054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a7244-aa7a-4bff-95c0-f7a9357ed2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:41.604277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a7244-aa7a-4bff-95c0-f7a9357ed2e7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:41.605546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 704/1395] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:41.611128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:41.611417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:41.611689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:41.628920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750cdae-78ca-467b-a1d1-3965027e7436 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:41.629202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750cdae-78ca-467b-a1d1-3965027e7436 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:41.629474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750cdae-78ca-467b-a1d1-3965027e7436 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:41.631528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 705/1396] 10.222.0.20 () {54 vars in 943 bytes} [Tue Sep 13 14:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:41.638133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:41.638481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:41.638822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:41.682343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:41.683475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:41.717176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] MFA Rules not processed for user `15e78b0e48a141edad05f3c064ac04f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:41.718419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:41.756185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:41.782847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 692/1397] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:41.831587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 706/1398] 10.222.0.20 () {58 vars in 1040 bytes} [Tue Sep 13 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:41.840914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:41.842164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:41.908548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:41.961859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d1dcb-c7ff-46d0-a5a6-bdd9fed3e8b2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:42.068715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee17455-6516-4d8d-ad58-fe96e68220c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:42.069308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee17455-6516-4d8d-ad58-fe96e68220c8 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:42.069588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee17455-6516-4d8d-ad58-fe96e68220c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:42.069887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee17455-6516-4d8d-ad58-fe96e68220c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:42.071534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6ee17455-6516-4d8d-ad58-fe96e68220c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:42.075995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:42.077219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:42.140743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:42.157524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee17455-6516-4d8d-ad58-fe96e68220c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:42.159715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee17455-6516-4d8d-ad58-fe96e68220c8 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:42.173184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 693/1399] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:05:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:05:42.178491 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0854620-634a-4cc8-b1a9-6b0684bad05a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:42.235938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:42.312753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d041d51-1a67-4b7b-b957-3bf9270b7cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:42.313597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d041d51-1a67-4b7b-b957-3bf9270b7cf4 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:42.313925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d041d51-1a67-4b7b-b957-3bf9270b7cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:42.314195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d041d51-1a67-4b7b-b957-3bf9270b7cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:42.316714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3d041d51-1a67-4b7b-b957-3bf9270b7cf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:42.423662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d041d51-1a67-4b7b-b957-3bf9270b7cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:42.425452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d041d51-1a67-4b7b-b957-3bf9270b7cf4 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:42.445502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 707/1400] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:05:44.716780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06324d-f061-4acb-908d-9a35ccf0c8dd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:44.717385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06324d-f061-4acb-908d-9a35ccf0c8dd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:44.717385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06324d-f061-4acb-908d-9a35ccf0c8dd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:44.721843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 694/1401] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:44.724346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4152fe-fd6a-491c-9594-f25742b04433 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:44.724810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4152fe-fd6a-491c-9594-f25742b04433 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:44.724904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4152fe-fd6a-491c-9594-f25742b04433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:44.797822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0b4152fe-fd6a-491c-9594-f25742b04433 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:44.798982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b4152fe-fd6a-491c-9594-f25742b04433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:44.807989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:44.909776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 708/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:45.201628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06324d-f061-4acb-908d-9a35ccf0c8dd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:45.203233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bc06324d-f061-4acb-908d-9a35ccf0c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:45.369208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06324d-f061-4acb-908d-9a35ccf0c8dd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:45.432400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:45.494241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e4ee8-7c24-4993-9b66-f297866b672e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:45.495129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4ee8-7c24-4993-9b66-f297866b672e service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:45.495748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4ee8-7c24-4993-9b66-f297866b672e service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:45.496276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e4ee8-7c24-4993-9b66-f297866b672e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:45.498009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c4e4ee8-7c24-4993-9b66-f297866b672e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:45.602147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4ee8-7c24-4993-9b66-f297866b672e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:45.603789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e4ee8-7c24-4993-9b66-f297866b672e service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:45.625984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 695/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:05:47.876899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae533b-fab4-40c7-ae49-9c77e31a6058 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:47.877457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae533b-fab4-40c7-ae49-9c77e31a6058 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:47.877457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae533b-fab4-40c7-ae49-9c77e31a6058 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:47.878927 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 709/1404] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:47.884879 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e913-3ab8-4718-a494-5ba575c3c4ce None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:47.885688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e913-3ab8-4718-a494-5ba575c3c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:47.886069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e913-3ab8-4718-a494-5ba575c3c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:47.958904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ef52e913-3ab8-4718-a494-5ba575c3c4ce None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:47.960142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ef52e913-3ab8-4718-a494-5ba575c3c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:47.980602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:48.070969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 696/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:48.366663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae533b-fab4-40c7-ae49-9c77e31a6058 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:48.368205 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b4ae533b-fab4-40c7-ae49-9c77e31a6058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:48.525774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:48.540957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae533b-fab4-40c7-ae49-9c77e31a6058 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:48.667762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dbb13-d636-4315-909c-e113cf9b5b6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:48.668349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dbb13-d636-4315-909c-e113cf9b5b6e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:48.668630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dbb13-d636-4315-909c-e113cf9b5b6e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:48.668923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dbb13-d636-4315-909c-e113cf9b5b6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:48.670604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fc0dbb13-d636-4315-909c-e113cf9b5b6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:48.772506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0dbb13-d636-4315-909c-e113cf9b5b6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:48.773245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0dbb13-d636-4315-909c-e113cf9b5b6e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:48.787879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:48.792578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 710/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:05:51.066340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5315cc-c61e-400e-b83a-aff42b15e302 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:51.066926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5315cc-c61e-400e-b83a-aff42b15e302 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:51.066926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5315cc-c61e-400e-b83a-aff42b15e302 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:51.069488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 697/1407] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:05:51.074638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d3441-1cff-4e4e-b452-9d2da08a5190 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:51.075068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d3441-1cff-4e4e-b452-9d2da08a5190 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:51.075309 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d3441-1cff-4e4e-b452-9d2da08a5190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:51.144234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0d1d3441-1cff-4e4e-b452-9d2da08a5190 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:05:51.145532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0d1d3441-1cff-4e4e-b452-9d2da08a5190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:51.261754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 711/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:05:51.554353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5315cc-c61e-400e-b83a-aff42b15e302 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:05:51.556220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5e5315cc-c61e-400e-b83a-aff42b15e302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:51.717934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5315cc-c61e-400e-b83a-aff42b15e302 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:05:51.720981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:05:51.848282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade7c2-307c-471a-a50f-725617d98f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:05:51.849861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade7c2-307c-471a-a50f-725617d98f20 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:05:51.850174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade7c2-307c-471a-a50f-725617d98f20 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:05:51.850462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade7c2-307c-471a-a50f-725617d98f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:05:51.852098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e9ade7c2-307c-471a-a50f-725617d98f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:05:51.964392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ade7c2-307c-471a-a50f-725617d98f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:05:51.966427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ade7c2-307c-471a-a50f-725617d98f20 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:05:51.985610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 698/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:06:01.260206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=66 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:01.986029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:05.079177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebbe2-59a4-4d12-907c-a58de136a8ed None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:05.079737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebbe2-59a4-4d12-907c-a58de136a8ed None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:05.079737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebbe2-59a4-4d12-907c-a58de136a8ed None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:05.081054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 712/1410] 10.222.0.20 () {54 vars in 918 bytes} [Tue Sep 13 14:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:05.087212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e075c9-1422-43be-9ec4-f6e374b89242 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:05.087722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e075c9-1422-43be-9ec4-f6e374b89242 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:05.087821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e075c9-1422-43be-9ec4-f6e374b89242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:05.163202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-50e075c9-1422-43be-9ec4-f6e374b89242 None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:05.164398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-50e075c9-1422-43be-9ec4-f6e374b89242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:05.268896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 699/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Sep 13 14:06:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:05.284390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b1bd4-366c-4b33-9844-52edaafc4468 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:05.284673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b1bd4-366c-4b33-9844-52edaafc4468 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:05.284947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b1bd4-366c-4b33-9844-52edaafc4468 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:05.286514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 713/1412] 10.222.0.20 () {54 vars in 918 bytes} [Tue Sep 13 14:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:05.291467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e715243-9f27-4d3b-8bd0-ec54453ebc39 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:05.291697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e715243-9f27-4d3b-8bd0-ec54453ebc39 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:05.291973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e715243-9f27-4d3b-8bd0-ec54453ebc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:05.364932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:05.367809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2e715243-9f27-4d3b-8bd0-ec54453ebc39 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:05.369537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2e715243-9f27-4d3b-8bd0-ec54453ebc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:05.485515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 700/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Sep 13 14:06:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:08.046804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd785bf2-841c-456c-9390-ae7e6da02d8d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:08.047366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd785bf2-841c-456c-9390-ae7e6da02d8d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:08.047366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd785bf2-841c-456c-9390-ae7e6da02d8d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:08.048661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:08.053966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecec28-f7bd-45f8-a64b-5b6e48416077 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:08.054441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecec28-f7bd-45f8-a64b-5b6e48416077 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:08.054541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecec28-f7bd-45f8-a64b-5b6e48416077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:08.120237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-93ecec28-f7bd-45f8-a64b-5b6e48416077 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:08.121444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-93ecec28-f7bd-45f8-a64b-5b6e48416077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:08.177069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:08.219270 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:08.509864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd785bf2-841c-456c-9390-ae7e6da02d8d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:08.511463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fd785bf2-841c-456c-9390-ae7e6da02d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:08.656645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd785bf2-841c-456c-9390-ae7e6da02d8d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:08.732535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:08.777194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991953b-925d-4532-bf19-82cc2587b440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:08.777749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991953b-925d-4532-bf19-82cc2587b440 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:08.778031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991953b-925d-4532-bf19-82cc2587b440 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:08.778302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991953b-925d-4532-bf19-82cc2587b440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:08.780163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c991953b-925d-4532-bf19-82cc2587b440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:08.869685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991953b-925d-4532-bf19-82cc2587b440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:08.871312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991953b-925d-4532-bf19-82cc2587b440 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:08.888916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 715/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:10.021016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862098-7ef6-4db2-9602-c4179148543a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:10.021754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862098-7ef6-4db2-9602-c4179148543a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:10.021754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862098-7ef6-4db2-9602-c4179148543a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:10.023103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:10.029644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b57c-c9c7-4424-a5c3-34bf4a91e791 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:10.030116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b57c-c9c7-4424-a5c3-34bf4a91e791 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:10.030259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b57c-c9c7-4424-a5c3-34bf4a91e791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:10.105592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1f48b57c-c9c7-4424-a5c3-34bf4a91e791 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:10.106828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1f48b57c-c9c7-4424-a5c3-34bf4a91e791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:10.133549 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:10.204056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:10.351534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb862098-7ef6-4db2-9602-c4179148543a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:10.353043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cb862098-7ef6-4db2-9602-c4179148543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:10.498051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb862098-7ef6-4db2-9602-c4179148543a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:10.539432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:10.614439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e56b65-878d-4f50-92e1-c858699ffcd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:10.614958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56b65-878d-4f50-92e1-c858699ffcd9 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:10.615232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56b65-878d-4f50-92e1-c858699ffcd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:10.615500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e56b65-878d-4f50-92e1-c858699ffcd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:10.617089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b6e56b65-878d-4f50-92e1-c858699ffcd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:10.706713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e56b65-878d-4f50-92e1-c858699ffcd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:10.708349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e56b65-878d-4f50-92e1-c858699ffcd9 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:10.726126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 703/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:11.855911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f43ed-131b-4466-95d1-300f8fb4e004 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:11.856308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f43ed-131b-4466-95d1-300f8fb4e004 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:11.856462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f43ed-131b-4466-95d1-300f8fb4e004 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:11.857890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:11.864928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc7e5-efca-41bb-8d79-fd7ffbaa7cd4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:11.865368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc7e5-efca-41bb-8d79-fd7ffbaa7cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:11.866078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc7e5-efca-41bb-8d79-fd7ffbaa7cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:11.923149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:11.945561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-adadc7e5-efca-41bb-8d79-fd7ffbaa7cd4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:11.946831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-adadc7e5-efca-41bb-8d79-fd7ffbaa7cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:12.042786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:12.196763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f43ed-131b-4466-95d1-300f8fb4e004 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:12.198261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7b1f43ed-131b-4466-95d1-300f8fb4e004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:12.346501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f43ed-131b-4466-95d1-300f8fb4e004 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:12.359482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:12.461388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d86e8-e854-44e9-8aea-06369d3ac428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:12.462029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d86e8-e854-44e9-8aea-06369d3ac428 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:12.462305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d86e8-e854-44e9-8aea-06369d3ac428 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:12.462576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d86e8-e854-44e9-8aea-06369d3ac428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:12.464181 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-084d86e8-e854-44e9-8aea-06369d3ac428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:12.554626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d86e8-e854-44e9-8aea-06369d3ac428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:12.556224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d86e8-e854-44e9-8aea-06369d3ac428 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:12.574615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 718/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:06:13.723008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa7d2-fcaa-41fe-8f21-bc687c490615 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:13.723580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa7d2-fcaa-41fe-8f21-bc687c490615 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:13.723580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa7d2-fcaa-41fe-8f21-bc687c490615 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:13.724949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:13.731340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e9897-f930-47d2-a8c8-253dd58f298f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:13.731774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e9897-f930-47d2-a8c8-253dd58f298f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:13.731890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e9897-f930-47d2-a8c8-253dd58f298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:13.752500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:13.808415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8d9e9897-f930-47d2-a8c8-253dd58f298f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:13.809602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d9e9897-f930-47d2-a8c8-253dd58f298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:13.902956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:14.054534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fa7d2-fcaa-41fe-8f21-bc687c490615 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:14.056030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-293fa7d2-fcaa-41fe-8f21-bc687c490615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:14.168632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:14.204923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fa7d2-fcaa-41fe-8f21-bc687c490615 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:14.317013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b80acc-3517-4360-a2bc-e12b1eba70e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:14.317526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80acc-3517-4360-a2bc-e12b1eba70e8 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:14.317833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80acc-3517-4360-a2bc-e12b1eba70e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:14.318104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80acc-3517-4360-a2bc-e12b1eba70e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:14.319687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-77b80acc-3517-4360-a2bc-e12b1eba70e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:14.395872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:14.409689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80acc-3517-4360-a2bc-e12b1eba70e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:14.411318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80acc-3517-4360-a2bc-e12b1eba70e8 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:14.429678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 706/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:15.549442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34f8e1-b238-4b70-ad15-54cac634d019 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:15.550005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34f8e1-b238-4b70-ad15-54cac634d019 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:15.550683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34f8e1-b238-4b70-ad15-54cac634d019 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:15.551911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:15.558171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b62d-b2f3-464a-9e97-378f31eeccd5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:15.558606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b62d-b2f3-464a-9e97-378f31eeccd5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:15.558707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550b62d-b2f3-464a-9e97-378f31eeccd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:15.634468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e550b62d-b2f3-464a-9e97-378f31eeccd5 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:15.635666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e550b62d-b2f3-464a-9e97-378f31eeccd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:15.731017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:15.880773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34f8e1-b238-4b70-ad15-54cac634d019 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:15.882311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4e34f8e1-b238-4b70-ad15-54cac634d019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:15.967233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:16.030166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34f8e1-b238-4b70-ad15-54cac634d019 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:16.142530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ea144-1c54-4f76-b43b-c83f015d93f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:16.143054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ea144-1c54-4f76-b43b-c83f015d93f6 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:16.143335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ea144-1c54-4f76-b43b-c83f015d93f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:16.143610 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ea144-1c54-4f76-b43b-c83f015d93f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:16.146067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c78ea144-1c54-4f76-b43b-c83f015d93f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:16.196424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:16.237975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ea144-1c54-4f76-b43b-c83f015d93f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:16.239714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ea144-1c54-4f76-b43b-c83f015d93f6 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:16.258407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 721/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:17.311804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daaeb3-4afa-4262-ad5f-52bcea35f1b7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:17.312318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daaeb3-4afa-4262-ad5f-52bcea35f1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:17.312424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daaeb3-4afa-4262-ad5f-52bcea35f1b7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:17.313690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:17.320789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed27fc0-9637-4071-a4ac-5f6bccf2bfe4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:17.321253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed27fc0-9637-4071-a4ac-5f6bccf2bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:17.321362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed27fc0-9637-4071-a4ac-5f6bccf2bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:17.400372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2ed27fc0-9637-4071-a4ac-5f6bccf2bfe4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:17.401671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2ed27fc0-9637-4071-a4ac-5f6bccf2bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:17.500163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:17.648992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daaeb3-4afa-4262-ad5f-52bcea35f1b7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:17.650463 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a8daaeb3-4afa-4262-ad5f-52bcea35f1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:17.699843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:17.795136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daaeb3-4afa-4262-ad5f-52bcea35f1b7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:17.907236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf98a76-961c-497a-9141-940a3c20b9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:17.907755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98a76-961c-497a-9141-940a3c20b9b9 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:17.908030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98a76-961c-497a-9141-940a3c20b9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:17.908297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf98a76-961c-497a-9141-940a3c20b9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:17.910449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9cf98a76-961c-497a-9141-940a3c20b9b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:17.917203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:17.999416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf98a76-961c-497a-9141-940a3c20b9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:18.001068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf98a76-961c-497a-9141-940a3c20b9b9 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:18.020120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 709/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:19.088520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa9062-5db6-4099-94a3-e8333d48091b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:19.088976 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa9062-5db6-4099-94a3-e8333d48091b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:19.089081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa9062-5db6-4099-94a3-e8333d48091b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:19.090516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:19.097382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d287229-4794-4f0a-8620-73813503d0cd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:19.097843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d287229-4794-4f0a-8620-73813503d0cd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:19.097966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d287229-4794-4f0a-8620-73813503d0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:19.173556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3d287229-4794-4f0a-8620-73813503d0cd None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:19.174870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3d287229-4794-4f0a-8620-73813503d0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:19.242712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:19.273318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:19.423445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa9062-5db6-4099-94a3-e8333d48091b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:19.424960 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-97fa9062-5db6-4099-94a3-e8333d48091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:19.443221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:19.572175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa9062-5db6-4099-94a3-e8333d48091b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:19.659885 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:19.685935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa11f68-99f5-4723-bc12-5716f1315cde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:19.686470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa11f68-99f5-4723-bc12-5716f1315cde service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:19.686745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa11f68-99f5-4723-bc12-5716f1315cde service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:19.687014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa11f68-99f5-4723-bc12-5716f1315cde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:19.688616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-efa11f68-99f5-4723-bc12-5716f1315cde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:19.776693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa11f68-99f5-4723-bc12-5716f1315cde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:19.778304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa11f68-99f5-4723-bc12-5716f1315cde service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:19.795888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 724/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:20.842524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05b0bb-168f-40e9-80a4-8a667bf1f99f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:20.843042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05b0bb-168f-40e9-80a4-8a667bf1f99f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:20.843042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05b0bb-168f-40e9-80a4-8a667bf1f99f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:20.844307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:20.851019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5868b6-6b3d-4b35-8e01-a593eefdbc28 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:20.851487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5868b6-6b3d-4b35-8e01-a593eefdbc28 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:20.851678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5868b6-6b3d-4b35-8e01-a593eefdbc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:20.928834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ff5868b6-6b3d-4b35-8e01-a593eefdbc28 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:20.930228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff5868b6-6b3d-4b35-8e01-a593eefdbc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:20.979509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:21.044598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:21.208856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05b0bb-168f-40e9-80a4-8a667bf1f99f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:21.210471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5b05b0bb-168f-40e9-80a4-8a667bf1f99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:21.360295 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05b0bb-168f-40e9-80a4-8a667bf1f99f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:21.424298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:21.489409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0370aa-193e-4bab-bdf3-f92505ca0d5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:21.489971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0370aa-193e-4bab-bdf3-f92505ca0d5d service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:21.490258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0370aa-193e-4bab-bdf3-f92505ca0d5d service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:21.490736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0370aa-193e-4bab-bdf3-f92505ca0d5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:21.493012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2b0370aa-193e-4bab-bdf3-f92505ca0d5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:21.591726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0370aa-193e-4bab-bdf3-f92505ca0d5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:21.593320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0370aa-193e-4bab-bdf3-f92505ca0d5d service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:21.614368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 712/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:22.673039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b480425-e387-4cd7-b478-87f04b4317c7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:22.673633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b480425-e387-4cd7-b478-87f04b4317c7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:22.673633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b480425-e387-4cd7-b478-87f04b4317c7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:22.675845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:22.681721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90de08-496a-46e3-b704-ad6128a05de8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:22.682163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90de08-496a-46e3-b704-ad6128a05de8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:22.682323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90de08-496a-46e3-b704-ad6128a05de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:22.758141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3e90de08-496a-46e3-b704-ad6128a05de8 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:22.759343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3e90de08-496a-46e3-b704-ad6128a05de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:22.761342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:22.857381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:23.011114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b480425-e387-4cd7-b478-87f04b4317c7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:23.012633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b480425-e387-4cd7-b478-87f04b4317c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:23.158067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b480425-e387-4cd7-b478-87f04b4317c7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:23.186357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:23.271243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912ca5c-3904-4dee-9769-1257ea08800e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:23.271842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912ca5c-3904-4dee-9769-1257ea08800e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:23.272117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912ca5c-3904-4dee-9769-1257ea08800e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:23.272381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912ca5c-3904-4dee-9769-1257ea08800e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:23.274004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5912ca5c-3904-4dee-9769-1257ea08800e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:23.364518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912ca5c-3904-4dee-9769-1257ea08800e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:23.366154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912ca5c-3904-4dee-9769-1257ea08800e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:23.384709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 727/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:25.490666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005c1a-8812-46db-b831-a96fbb10ebe3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:25.491224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005c1a-8812-46db-b831-a96fbb10ebe3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:25.491224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005c1a-8812-46db-b831-a96fbb10ebe3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:25.492551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:25.497897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032ba0c-3e48-43d9-923b-7fcbf652482d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:25.498279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032ba0c-3e48-43d9-923b-7fcbf652482d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:25.498447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032ba0c-3e48-43d9-923b-7fcbf652482d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:25.540268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:25.567168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1032ba0c-3e48-43d9-923b-7fcbf652482d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:25.568387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1032ba0c-3e48-43d9-923b-7fcbf652482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:25.665399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:25.956242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc005c1a-8812-46db-b831-a96fbb10ebe3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:25.957891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cc005c1a-8812-46db-b831-a96fbb10ebe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:26.062559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc005c1a-8812-46db-b831-a96fbb10ebe3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:26.094288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:26.137801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81feb-36c2-4160-9263-9176f56a10b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:26.138400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81feb-36c2-4160-9263-9176f56a10b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:26.138677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81feb-36c2-4160-9263-9176f56a10b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:26.138947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81feb-36c2-4160-9263-9176f56a10b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:26.140540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-71d81feb-36c2-4160-9263-9176f56a10b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:26.369050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d81feb-36c2-4160-9263-9176f56a10b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:26.370434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d81feb-36c2-4160-9263-9176f56a10b2 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:26.388910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 715/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:28.667647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b5caf-aa84-4279-bfe1-94153f3274f6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:28.668182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b5caf-aa84-4279-bfe1-94153f3274f6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:28.668182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b5caf-aa84-4279-bfe1-94153f3274f6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:28.669565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:28.675187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82305d9e-18ce-40e0-8cc4-36994f25fdfd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:28.675623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82305d9e-18ce-40e0-8cc4-36994f25fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:28.675736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82305d9e-18ce-40e0-8cc4-36994f25fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:28.741865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-82305d9e-18ce-40e0-8cc4-36994f25fdfd None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:28.743077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-82305d9e-18ce-40e0-8cc4-36994f25fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:28.755980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:28.840499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:06:29.133711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b5caf-aa84-4279-bfe1-94153f3274f6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:29.135291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-747b5caf-aa84-4279-bfe1-94153f3274f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:29.237472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b5caf-aa84-4279-bfe1-94153f3274f6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:29.249998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:29.310772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287797e-c8e6-4a12-9896-e198ac2723ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:29.311369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287797e-c8e6-4a12-9896-e198ac2723ce service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:29.311641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287797e-c8e6-4a12-9896-e198ac2723ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:29.311910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287797e-c8e6-4a12-9896-e198ac2723ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:29.313495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f287797e-c8e6-4a12-9896-e198ac2723ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:29.421589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287797e-c8e6-4a12-9896-e198ac2723ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:29.422956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287797e-c8e6-4a12-9896-e198ac2723ce service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:29.441644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 730/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:31.979868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e13e1d-7bed-4865-a38c-a9c48c8325c5 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:31.980422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e13e1d-7bed-4865-a38c-a9c48c8325c5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:31.980422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e13e1d-7bed-4865-a38c-a9c48c8325c5 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:31.981758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:31.988071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67fef7-8a04-42e8-8737-d0ccddf28ebf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:31.988501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67fef7-8a04-42e8-8737-d0ccddf28ebf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:31.988614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67fef7-8a04-42e8-8737-d0ccddf28ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:32.037168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:32.056304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8d67fef7-8a04-42e8-8737-d0ccddf28ebf None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:32.057517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8d67fef7-8a04-42e8-8737-d0ccddf28ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:32.154546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:32.438831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e13e1d-7bed-4865-a38c-a9c48c8325c5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:32.440553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d0e13e1d-7bed-4865-a38c-a9c48c8325c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:32.544004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e13e1d-7bed-4865-a38c-a9c48c8325c5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:32.586173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:32.621489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70b761-7d7c-470c-b093-c9f3caa9d4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:32.622119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70b761-7d7c-470c-b093-c9f3caa9d4db service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:32.622394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70b761-7d7c-470c-b093-c9f3caa9d4db service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:32.622661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70b761-7d7c-470c-b093-c9f3caa9d4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:32.624245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f70b761-7d7c-470c-b093-c9f3caa9d4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:32.731279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70b761-7d7c-470c-b093-c9f3caa9d4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:32.732745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70b761-7d7c-470c-b093-c9f3caa9d4db service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:32.750238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 718/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:35.067681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0a8ba-4328-4be5-8ae1-0aed3b353d94 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:35.068127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0a8ba-4328-4be5-8ae1-0aed3b353d94 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:35.068279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0a8ba-4328-4be5-8ae1-0aed3b353d94 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:35.069632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:35.074931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74c069-9d5b-4ba4-b74a-23ef41df7114 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:35.075330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74c069-9d5b-4ba4-b74a-23ef41df7114 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:35.075483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74c069-9d5b-4ba4-b74a-23ef41df7114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:35.141332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fc74c069-9d5b-4ba4-b74a-23ef41df7114 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:35.142590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc74c069-9d5b-4ba4-b74a-23ef41df7114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:35.164001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:35.240615 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:35.527229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0a8ba-4328-4be5-8ae1-0aed3b353d94 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:35.528474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-feb0a8ba-4328-4be5-8ae1-0aed3b353d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:35.629961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0a8ba-4328-4be5-8ae1-0aed3b353d94 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:35.650223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:35.703252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818d003-3fea-4ddd-a90c-eb0c74063a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:35.703845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818d003-3fea-4ddd-a90c-eb0c74063a8a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:35.704122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818d003-3fea-4ddd-a90c-eb0c74063a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:35.704387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818d003-3fea-4ddd-a90c-eb0c74063a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:35.706697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5818d003-3fea-4ddd-a90c-eb0c74063a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:35.812928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818d003-3fea-4ddd-a90c-eb0c74063a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:35.814303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818d003-3fea-4ddd-a90c-eb0c74063a8a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:35.833213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 733/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:38.019428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585e438-c48f-444a-9f18-7feec2af72bd None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:38.019986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585e438-c48f-444a-9f18-7feec2af72bd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:38.019986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585e438-c48f-444a-9f18-7feec2af72bd None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:38.021296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:38.026577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535e267-a26b-41d1-8b78-93aa7686ec60 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:38.026977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535e267-a26b-41d1-8b78-93aa7686ec60 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:38.027135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535e267-a26b-41d1-8b78-93aa7686ec60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:38.091371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:38.094485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8535e267-a26b-41d1-8b78-93aa7686ec60 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:38.096346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8535e267-a26b-41d1-8b78-93aa7686ec60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:38.195596 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:38.476611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585e438-c48f-444a-9f18-7feec2af72bd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:38.478190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c585e438-c48f-444a-9f18-7feec2af72bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:38.583176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585e438-c48f-444a-9f18-7feec2af72bd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:38.633243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:38.657800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a681-aecf-487f-92b4-21a367c3b5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:38.659018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a681-aecf-487f-92b4-21a367c3b5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:38.659292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a681-aecf-487f-92b4-21a367c3b5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:38.659557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a681-aecf-487f-92b4-21a367c3b5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:38.661141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-aa20a681-aecf-487f-92b4-21a367c3b5e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:38.771189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a681-aecf-487f-92b4-21a367c3b5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:38.772562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a681-aecf-487f-92b4-21a367c3b5e6 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:38.791409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 721/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:40.979513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb5928-a4ab-49e9-bc84-4da98ea4600e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:40.980064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb5928-a4ab-49e9-bc84-4da98ea4600e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:40.980064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb5928-a4ab-49e9-bc84-4da98ea4600e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:40.981485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:40.986872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2474d-26e8-447c-b746-9d1e8a0ae24b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:40.987835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2474d-26e8-447c-b746-9d1e8a0ae24b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:40.988031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2474d-26e8-447c-b746-9d1e8a0ae24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:41.054101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9fe2474d-26e8-447c-b746-9d1e8a0ae24b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:41.055316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9fe2474d-26e8-447c-b746-9d1e8a0ae24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:41.083887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:41.149860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:41.434562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb5928-a4ab-49e9-bc84-4da98ea4600e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:41.436094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-11cb5928-a4ab-49e9-bc84-4da98ea4600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:41.538652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb5928-a4ab-49e9-bc84-4da98ea4600e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:41.569246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:41.614413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f61f3-8fe4-4132-ae3f-1306ca072d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:41.615006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f61f3-8fe4-4132-ae3f-1306ca072d4f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:41.615280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f61f3-8fe4-4132-ae3f-1306ca072d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:41.615545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f61f3-8fe4-4132-ae3f-1306ca072d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:41.617132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d01f61f3-8fe4-4132-ae3f-1306ca072d4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:41.726823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f61f3-8fe4-4132-ae3f-1306ca072d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:41.728296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f61f3-8fe4-4132-ae3f-1306ca072d4f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:41.747040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 736/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:43.947295 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07206d83-2747-4444-abbb-18192d8549db None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:43.947843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07206d83-2747-4444-abbb-18192d8549db None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:43.947843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07206d83-2747-4444-abbb-18192d8549db None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:43.949076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:43.954503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd449e-22cc-4b68-b364-f353b5ffee6c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:43.955580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd449e-22cc-4b68-b364-f353b5ffee6c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:43.955681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd449e-22cc-4b68-b364-f353b5ffee6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:44.019854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-68fd449e-22cc-4b68-b364-f353b5ffee6c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:44.021077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68fd449e-22cc-4b68-b364-f353b5ffee6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:44.029637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:44.120082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:06:44.424371 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07206d83-2747-4444-abbb-18192d8549db None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:44.425964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07206d83-2747-4444-abbb-18192d8549db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:44.528575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07206d83-2747-4444-abbb-18192d8549db None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:44.584397 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:44.602264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263a00-940d-439e-888d-c514cb28264c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:44.602873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263a00-940d-439e-888d-c514cb28264c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:44.603153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263a00-940d-439e-888d-c514cb28264c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:44.603428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263a00-940d-439e-888d-c514cb28264c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:44.605106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-01263a00-940d-439e-888d-c514cb28264c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:44.713743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01263a00-940d-439e-888d-c514cb28264c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:44.715175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01263a00-940d-439e-888d-c514cb28264c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:44.732841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 724/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:46.925543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0552-0eeb-49d4-bda6-23e1c876b298 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:46.926177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0552-0eeb-49d4-bda6-23e1c876b298 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:46.926177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0552-0eeb-49d4-bda6-23e1c876b298 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:46.927370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:46.932616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844cf97-3286-4e01-ad8e-43572ae7fe26 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:46.933075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844cf97-3286-4e01-ad8e-43572ae7fe26 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:46.933185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844cf97-3286-4e01-ad8e-43572ae7fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:46.999617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4844cf97-3286-4e01-ad8e-43572ae7fe26 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:47.000824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4844cf97-3286-4e01-ad8e-43572ae7fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:47.040408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:47.098473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:47.385577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0552-0eeb-49d4-bda6-23e1c876b298 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:47.387222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f0d0552-0eeb-49d4-bda6-23e1c876b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:47.492336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0552-0eeb-49d4-bda6-23e1c876b298 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:47.532305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:47.569461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f59dc5-3715-4cc9-a1c3-9039d50067f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:47.570099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f59dc5-3715-4cc9-a1c3-9039d50067f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:47.570379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f59dc5-3715-4cc9-a1c3-9039d50067f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:47.570651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f59dc5-3715-4cc9-a1c3-9039d50067f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:47.572245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-61f59dc5-3715-4cc9-a1c3-9039d50067f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:47.683686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f59dc5-3715-4cc9-a1c3-9039d50067f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:47.685110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f59dc5-3715-4cc9-a1c3-9039d50067f5 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:47.703175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 739/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:49.834410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470cf4-6e3b-4ed9-aa45-83b68e72db1f None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:49.834944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470cf4-6e3b-4ed9-aa45-83b68e72db1f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:49.834944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470cf4-6e3b-4ed9-aa45-83b68e72db1f None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:49.836183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:49.841382 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3772c77-e440-402b-9f9e-6e8fca5bf9cb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:49.841824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3772c77-e440-402b-9f9e-6e8fca5bf9cb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:49.841966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3772c77-e440-402b-9f9e-6e8fca5bf9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:49.905788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b3772c77-e440-402b-9f9e-6e8fca5bf9cb None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:49.906970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b3772c77-e440-402b-9f9e-6e8fca5bf9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:49.925058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:50.002694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:50.287912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef470cf4-6e3b-4ed9-aa45-83b68e72db1f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:50.289450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ef470cf4-6e3b-4ed9-aa45-83b68e72db1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:50.392449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef470cf4-6e3b-4ed9-aa45-83b68e72db1f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:50.459074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:50.465675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b860e6-62b8-4c2b-9a90-8c81c7cde620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:50.466303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b860e6-62b8-4c2b-9a90-8c81c7cde620 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:50.466580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b860e6-62b8-4c2b-9a90-8c81c7cde620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:50.466849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b860e6-62b8-4c2b-9a90-8c81c7cde620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:50.468437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a8b860e6-62b8-4c2b-9a90-8c81c7cde620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:50.579398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b860e6-62b8-4c2b-9a90-8c81c7cde620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:50.580764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b860e6-62b8-4c2b-9a90-8c81c7cde620 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:50.606106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 727/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:06:52.805091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7e9f-e476-4995-bd5b-df8dc643cebe None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:52.805626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7e9f-e476-4995-bd5b-df8dc643cebe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:52.805626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7e9f-e476-4995-bd5b-df8dc643cebe None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:52.806922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:06:52.812129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44d4bb-530c-4896-8817-ba88201c312c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:52.812619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44d4bb-530c-4896-8817-ba88201c312c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:52.812722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44d4bb-530c-4896-8817-ba88201c312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:52.879434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7c44d4bb-530c-4896-8817-ba88201c312c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:06:52.880632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c44d4bb-530c-4896-8817-ba88201c312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:52.929608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:52.978382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:06:53.262275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d7e9f-e476-4995-bd5b-df8dc643cebe None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:06:53.263819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-772d7e9f-e476-4995-bd5b-df8dc643cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:53.365950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d7e9f-e476-4995-bd5b-df8dc643cebe None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:06:53.414025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:06:53.441520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ee0b4-a300-4c07-9fa2-fc3c2272a09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:06:53.442147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee0b4-a300-4c07-9fa2-fc3c2272a09b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:06:53.442421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee0b4-a300-4c07-9fa2-fc3c2272a09b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:06:53.442688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee0b4-a300-4c07-9fa2-fc3c2272a09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:06:53.444279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-159ee0b4-a300-4c07-9fa2-fc3c2272a09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:06:53.552672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ee0b4-a300-4c07-9fa2-fc3c2272a09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:06:53.554144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ee0b4-a300-4c07-9fa2-fc3c2272a09b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:06:53.572563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 742/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:07:02.976812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=22 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:03.570341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=66 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:13.424332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104a43-1471-44bf-8039-c5ddd80a35a3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:13.424863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104a43-1471-44bf-8039-c5ddd80a35a3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:13.424863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104a43-1471-44bf-8039-c5ddd80a35a3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:13.426172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 729/1471] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:13.431487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066dff1-2f74-4e7c-94fe-d17c92d3dbbb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:13.431909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066dff1-2f74-4e7c-94fe-d17c92d3dbbb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:13.432040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066dff1-2f74-4e7c-94fe-d17c92d3dbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:13.497031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2066dff1-2f74-4e7c-94fe-d17c92d3dbbb None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:13.498254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2066dff1-2f74-4e7c-94fe-d17c92d3dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:13.594909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 743/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:13.893824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8811f-3c2d-40a4-abcc-8ace2f183b2a None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:13.894116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8811f-3c2d-40a4-abcc-8ace2f183b2a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:13.894417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8811f-3c2d-40a4-abcc-8ace2f183b2a None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:13.895733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 730/1473] 10.222.0.20 () {54 vars in 966 bytes} [Tue Sep 13 14:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:13.900663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7d69-5435-49cc-a671-43e293225197 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:13.900946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7d69-5435-49cc-a671-43e293225197 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:13.901218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7d69-5435-49cc-a671-43e293225197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:13.976980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8ebd7d69-5435-49cc-a671-43e293225197 None None] MFA Rules not processed for user `f7214e6addc646a899efb82283ededc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:13.978210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8ebd7d69-5435-49cc-a671-43e293225197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:13.998808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:14.054267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 744/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Tue Sep 13 14:07:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:14.061935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8811f-3c2d-40a4-abcc-8ace2f183b2a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:07:14.064158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fdf8811f-3c2d-40a4-abcc-8ace2f183b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:14.169041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8811f-3c2d-40a4-abcc-8ace2f183b2a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:07:14.242880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629f81c-abe4-47e8-a93b-d67d15a3f565 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7214e6addc646a899efb82283ededc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:07:14.243474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f81c-abe4-47e8-a93b-d67d15a3f565 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:14.243748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f81c-abe4-47e8-a93b-d67d15a3f565 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:14.244019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f81c-abe4-47e8-a93b-d67d15a3f565 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:14.245593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e629f81c-abe4-47e8-a93b-d67d15a3f565 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:14.299939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:14.354964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629f81c-abe4-47e8-a93b-d67d15a3f565 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:07:14.356302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629f81c-abe4-47e8-a93b-d67d15a3f565 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:07:14.375428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 731/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:07:16.610878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a04a2-137f-43f7-80e6-4d791ed8c01f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:16.611400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a04a2-137f-43f7-80e6-4d791ed8c01f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:16.611400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a04a2-137f-43f7-80e6-4d791ed8c01f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:16.612658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 745/1476] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:16.617865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662eb21-56d8-4f0f-abc6-19f13f08d81a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:16.618275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662eb21-56d8-4f0f-abc6-19f13f08d81a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:16.618415 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662eb21-56d8-4f0f-abc6-19f13f08d81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:16.683584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4662eb21-56d8-4f0f-abc6-19f13f08d81a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:16.684785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4662eb21-56d8-4f0f-abc6-19f13f08d81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:16.779621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 732/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:17.051538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a04a2-137f-43f7-80e6-4d791ed8c01f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:07:17.053114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6a1a04a2-137f-43f7-80e6-4d791ed8c01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:17.183536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a04a2-137f-43f7-80e6-4d791ed8c01f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:07:17.352982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:17.399331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9135c-d532-4b3c-8a23-1e84ac59ade4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:07:17.399944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9135c-d532-4b3c-8a23-1e84ac59ade4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:17.400221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9135c-d532-4b3c-8a23-1e84ac59ade4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:17.400492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9135c-d532-4b3c-8a23-1e84ac59ade4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:17.401140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9135c-d532-4b3c-8a23-1e84ac59ade4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:07:17.403166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-b2e9135c-d532-4b3c-8a23-1e84ac59ade4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:07:17.403166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:07:17.403166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:07:17.404303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9135c-d532-4b3c-8a23-1e84ac59ade4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:07:17.425118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 746/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Sep 13 14:07:17 2022] GET /identity/v3/projects => generated 2553 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:07:19.600943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8e32e-e934-4d22-a66a-e5cdb77b8107 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:19.601501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8e32e-e934-4d22-a66a-e5cdb77b8107 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:19.601501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8e32e-e934-4d22-a66a-e5cdb77b8107 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:19.602852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 733/1479] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:19.608071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c7f1e-7199-4430-8ae1-de9de5b25d6a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:19.608468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c7f1e-7199-4430-8ae1-de9de5b25d6a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:19.608626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c7f1e-7199-4430-8ae1-de9de5b25d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:19.675320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0b5c7f1e-7199-4430-8ae1-de9de5b25d6a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:19.676522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b5c7f1e-7199-4430-8ae1-de9de5b25d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:19.770331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 747/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:20.065640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8e32e-e934-4d22-a66a-e5cdb77b8107 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:07:20.067244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8fc8e32e-e934-4d22-a66a-e5cdb77b8107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:20.076990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:20.213747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8e32e-e934-4d22-a66a-e5cdb77b8107 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:07:20.294801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:20.328051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9514f-56ad-43e7-89cb-438ed2ae2ff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:07:20.328738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9514f-56ad-43e7-89cb-438ed2ae2ff8 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:20.329014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9514f-56ad-43e7-89cb-438ed2ae2ff8 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:20.329281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9514f-56ad-43e7-89cb-438ed2ae2ff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:20.330902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-68a9514f-56ad-43e7-89cb-438ed2ae2ff8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:20.421161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9514f-56ad-43e7-89cb-438ed2ae2ff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:07:20.422934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9514f-56ad-43e7-89cb-438ed2ae2ff8 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:07:20.440676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 734/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:07:22.846176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732f7e-c3e9-4e70-aff2-c84ea88b13b9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:22.846732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732f7e-c3e9-4e70-aff2-c84ea88b13b9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:22.846732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04732f7e-c3e9-4e70-aff2-c84ea88b13b9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:22.848030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 748/1482] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:22.853573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31345e-fbcd-402b-8d1a-671ed55ad147 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:22.854068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31345e-fbcd-402b-8d1a-671ed55ad147 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:22.854176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31345e-fbcd-402b-8d1a-671ed55ad147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:22.928686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4d31345e-fbcd-402b-8d1a-671ed55ad147 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:22.929943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4d31345e-fbcd-402b-8d1a-671ed55ad147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:22.987206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:23.046743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 735/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:23.343532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732f7e-c3e9-4e70-aff2-c84ea88b13b9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:07:23.345168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-04732f7e-c3e9-4e70-aff2-c84ea88b13b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:23.369052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:23.448763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04732f7e-c3e9-4e70-aff2-c84ea88b13b9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:07:23.523182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b717a15-ee85-4f14-a750-eec369364d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:07:23.523783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717a15-ee85-4f14-a750-eec369364d21 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:23.524056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717a15-ee85-4f14-a750-eec369364d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:23.524323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717a15-ee85-4f14-a750-eec369364d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:23.525956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7b717a15-ee85-4f14-a750-eec369364d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:23.600605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:23.635013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b717a15-ee85-4f14-a750-eec369364d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:07:23.636444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b717a15-ee85-4f14-a750-eec369364d21 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:07:23.654576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 749/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:07:25.853112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709ad03-e8be-401c-bb03-f4de67c1bf94 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:25.853646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709ad03-e8be-401c-bb03-f4de67c1bf94 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:25.853646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709ad03-e8be-401c-bb03-f4de67c1bf94 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:25.855003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 736/1485] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:25.860685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d84e4-81e5-48cb-9a7a-604f557360c0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:25.861072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d84e4-81e5-48cb-9a7a-604f557360c0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:25.861233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d84e4-81e5-48cb-9a7a-604f557360c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:25.928373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-142d84e4-81e5-48cb-9a7a-604f557360c0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:25.929643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-142d84e4-81e5-48cb-9a7a-604f557360c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:26.026537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 750/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:26.315858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709ad03-e8be-401c-bb03-f4de67c1bf94 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:07:26.317425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6709ad03-e8be-401c-bb03-f4de67c1bf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:26.423141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709ad03-e8be-401c-bb03-f4de67c1bf94 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:07:26.493970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:26.495572 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed547a-9f70-4b7a-87d7-64253fba165a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:07:26.496256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed547a-9f70-4b7a-87d7-64253fba165a service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:26.496756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed547a-9f70-4b7a-87d7-64253fba165a service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:26.497183 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed547a-9f70-4b7a-87d7-64253fba165a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:26.499101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-08ed547a-9f70-4b7a-87d7-64253fba165a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:26.609220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed547a-9f70-4b7a-87d7-64253fba165a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:07:26.610639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed547a-9f70-4b7a-87d7-64253fba165a service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:07:26.628111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 737/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:07:28.871230 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98583cec-996f-409e-8271-cf2fd872073e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:28.871770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98583cec-996f-409e-8271-cf2fd872073e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:28.871770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98583cec-996f-409e-8271-cf2fd872073e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:28.873004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 751/1488] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:28.878785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1590ac-e780-447b-b84c-c68e8d2286b0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:28.879216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1590ac-e780-447b-b84c-c68e8d2286b0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:28.879358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1590ac-e780-447b-b84c-c68e8d2286b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:28.944868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fc1590ac-e780-447b-b84c-c68e8d2286b0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:28.946085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc1590ac-e780-447b-b84c-c68e8d2286b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:28.997688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:29.042795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 738/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:29.333878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98583cec-996f-409e-8271-cf2fd872073e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:07:29.335430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-98583cec-996f-409e-8271-cf2fd872073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:29.371476 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:29.438764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98583cec-996f-409e-8271-cf2fd872073e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:07:29.510965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6c2df-3b37-4f42-9a54-09915ea5dc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:07:29.511626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6c2df-3b37-4f42-9a54-09915ea5dc50 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:29.511908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6c2df-3b37-4f42-9a54-09915ea5dc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:29.512176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6c2df-3b37-4f42-9a54-09915ea5dc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:29.513787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8ad6c2df-3b37-4f42-9a54-09915ea5dc50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:29.593582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:29.623575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6c2df-3b37-4f42-9a54-09915ea5dc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:07:29.626019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6c2df-3b37-4f42-9a54-09915ea5dc50 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:07:29.644714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 752/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:07:32.072288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767990-fad5-42a5-abc5-0bfe7ffb0e10 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:32.072857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767990-fad5-42a5-abc5-0bfe7ffb0e10 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:32.072857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767990-fad5-42a5-abc5-0bfe7ffb0e10 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:32.074211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 739/1491] 10.222.0.20 () {54 vars in 927 bytes} [Tue Sep 13 14:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:07:32.080238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf34d40-b96f-45d0-a4fe-3b8ce9438b57 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:32.080658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf34d40-b96f-45d0-a4fe-3b8ce9438b57 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:32.080790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf34d40-b96f-45d0-a4fe-3b8ce9438b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:32.145452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dbf34d40-b96f-45d0-a4fe-3b8ce9438b57 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:07:32.146680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dbf34d40-b96f-45d0-a4fe-3b8ce9438b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:32.243766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 753/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Tue Sep 13 14:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:07:32.531562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6767990-fad5-42a5-abc5-0bfe7ffb0e10 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:07:32.533089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d6767990-fad5-42a5-abc5-0bfe7ffb0e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:32.636064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6767990-fad5-42a5-abc5-0bfe7ffb0e10 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:07:32.708195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201593a-24e6-4f29-9118-1b6bb4e68656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:07:32.708794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201593a-24e6-4f29-9118-1b6bb4e68656 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:07:32.709070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201593a-24e6-4f29-9118-1b6bb4e68656 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:07:32.709339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201593a-24e6-4f29-9118-1b6bb4e68656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:07:32.710953 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c201593a-24e6-4f29-9118-1b6bb4e68656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:07:32.719893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:32.818040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201593a-24e6-4f29-9118-1b6bb4e68656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:07:32.819388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201593a-24e6-4f29-9118-1b6bb4e68656 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:07:32.836938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 740/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:07:42.242328 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=82 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:07:42.834823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=50 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:45.049626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:45.050271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:45.050271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:45.127002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:08:45.128202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:45.243585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 754/1494] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:08:45.250994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc7294-209e-4d13-badf-d81ebfa4b448 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:45.251539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc7294-209e-4d13-badf-d81ebfa4b448 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:45.251539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc7294-209e-4d13-badf-d81ebfa4b448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:45.327862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-54cc7294-209e-4d13-badf-d81ebfa4b448 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:08:45.328992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-54cc7294-209e-4d13-badf-d81ebfa4b448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:45.425879 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 741/1495] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:08:45.430550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:45.431756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:45.443759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb5b39-6c66-4988-ab31-569bc1f04a38 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:45.504078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:45.539758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:45.540364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:45.540642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:45.540911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:45.541555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:08:45.541887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:45.542898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:45.542898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:08:45.542898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:08:45.543818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:45.554931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 755/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:08:45.563828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc7294-209e-4d13-badf-d81ebfa4b448 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:45.573545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc7294-209e-4d13-badf-d81ebfa4b448 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:45.634871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:45.668914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:45.669506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:45.669809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:45.670083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:45.670715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:45.672440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:45.672440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Sep 13 14:08:45.672440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: Sep 13 14:08:45.673621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:45.728591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 742/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:08:45 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:08:45.732871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:45.734406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:45.767572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5af0d-1d60-45e8-8e17-77b9edb05319 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:45.861911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:45.862513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:45.862790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:45.863058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:45.863682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:45.864690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:45.864690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:08:45.864690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:08:45.865921 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:45.907745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 756/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:08:45 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:08:45.911812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:45.913338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:45.975527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938c411-699f-4781-87be-893b24b823bd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:46.059961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:46.070892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:46.071492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:46.071768 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:46.072035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:46.072672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:46.073791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:46.073791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Sep 13 14:08:46.073791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: Sep 13 14:08:46.074673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:46.086825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 743/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:08:46.090946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:46.092467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:46.103709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da07e8-280a-4c2a-9868-65dc6f922ea6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:46.129272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:46.199943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95db7d-34c3-4faf-acca-c49129126e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:46.200564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95db7d-34c3-4faf-acca-c49129126e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:46.200837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95db7d-34c3-4faf-acca-c49129126e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:46.201101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95db7d-34c3-4faf-acca-c49129126e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:46.201750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95db7d-34c3-4faf-acca-c49129126e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:46.202755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-cc95db7d-34c3-4faf-acca-c49129126e04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:46.202755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:08:46.202755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:08:46.203637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95db7d-34c3-4faf-acca-c49129126e04 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:46.216181 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 757/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:08:46.220329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:46.221876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:46.257098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1569dd-8339-4d6a-b929-562f4bc576b0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:46.349154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07256d-6017-4a78-a3af-6d13e2257e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:46.349825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07256d-6017-4a78-a3af-6d13e2257e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:46.350103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07256d-6017-4a78-a3af-6d13e2257e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:46.350370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07256d-6017-4a78-a3af-6d13e2257e09 admin admin] PATH_INFO: `/v3/projects/3dcbf6efa92344d19e35a42b914d692c/users/1b6bfa4c91634450a0b90b4fd151fa0c/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:46.374010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07256d-6017-4a78-a3af-6d13e2257e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dcbf6efa92344d19e35a42b914d692c, user_id=1b6bfa4c91634450a0b90b4fd151fa0c, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:46.375432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-dd07256d-6017-4a78-a3af-6d13e2257e09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:46.375432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Sep 13 14:08:46.375432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: Sep 13 14:08:46.377347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07256d-6017-4a78-a3af-6d13e2257e09 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:46.402580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 744/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:08:46 2022] PUT /identity/v3/projects/3dcbf6efa92344d19e35a42b914d692c/users/1b6bfa4c91634450a0b90b4fd151fa0c/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:08:46.414638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f2544-3142-4190-94bf-a08360df1dff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:46.414930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f2544-3142-4190-94bf-a08360df1dff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:46.415211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f2544-3142-4190-94bf-a08360df1dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:46.472772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a30f2544-3142-4190-94bf-a08360df1dff None None] MFA Rules not processed for user `1b6bfa4c91634450a0b90b4fd151fa0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:08:46.473958 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a30f2544-3142-4190-94bf-a08360df1dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:46.551925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 758/1502] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:08:46.565686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881fa5-005c-483c-bc7b-394c367144e0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:46.566005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881fa5-005c-483c-bc7b-394c367144e0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:46.566305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881fa5-005c-483c-bc7b-394c367144e0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:46.567660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 745/1503] 10.222.0.20 () {54 vars in 964 bytes} [Tue Sep 13 14:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:08:46.572577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedaf40-f5ac-4410-9ab6-a6657ef15e12 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:46.572875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedaf40-f5ac-4410-9ab6-a6657ef15e12 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:46.573389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedaf40-f5ac-4410-9ab6-a6657ef15e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:46.604894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:46.649379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1aedaf40-f5ac-4410-9ab6-a6657ef15e12 None None] MFA Rules not processed for user `c95553acbf23423993a4d25e83ecee48`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:08:46.650938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1aedaf40-f5ac-4410-9ab6-a6657ef15e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:46.725739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 759/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Sep 13 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:08:46.735979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881fa5-005c-483c-bc7b-394c367144e0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:46.737503 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f6881fa5-005c-483c-bc7b-394c367144e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:46.788154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:46.841932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881fa5-005c-483c-bc7b-394c367144e0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:46.918720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:46.919318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:46.919594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:46.919863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:46.921455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:47.017047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:47.018472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:47.021969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:47.037088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 746/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:08:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:08:47.082484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4997d-7c4b-4202-a99c-dea59c02d521 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:47.082773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4997d-7c4b-4202-a99c-dea59c02d521 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:47.083047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4997d-7c4b-4202-a99c-dea59c02d521 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:47.084332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 760/1506] 10.222.0.20 () {52 vars in 830 bytes} [Tue Sep 13 14:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:08:47.087933 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:47.089880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:47.191285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f92a2-3f9d-4d03-9693-12b400c3626a service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:47.286446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:47.287084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:47.287245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:47.287514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] PATH_INFO: `/v3/users/1b6bfa4c91634450a0b90b4fd151fa0c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:47.301185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b6bfa4c91634450a0b90b4fd151fa0c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:47.302245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:47.302245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Sep 13 14:08:47.302245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: Sep 13 14:08:47.303451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:47.314157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:47.369308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b6bfa4c91634450a0b90b4fd151fa0c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:47.384932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b6bfa4c91634450a0b90b4fd151fa0c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:47.410467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b6bfa4c91634450a0b90b4fd151fa0c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:47.411093 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b6bfa4c91634450a0b90b4fd151fa0c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:47.411557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b6bfa4c91634450a0b90b4fd151fa0c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:47.428488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 747/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:08:47 2022] DELETE /identity/v3/users/1b6bfa4c91634450a0b90b4fd151fa0c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:08:47.433682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:47.434187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:47.434287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:47.490734 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:08:47.491928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:47.569021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:47.581602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 761/1508] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:08:47.591091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:47.592357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:47.692608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e107a-7fa2-487a-b1fa-1ee973d3101b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:47.748593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:47.768582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4d077-82df-475d-a60e-ada549e40bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:47.769189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d077-82df-475d-a60e-ada549e40bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:47.769469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d077-82df-475d-a60e-ada549e40bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:47.769770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d077-82df-475d-a60e-ada549e40bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:47.771369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b0c4d077-82df-475d-a60e-ada549e40bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:47.877924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4d077-82df-475d-a60e-ada549e40bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:47.879272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4d077-82df-475d-a60e-ada549e40bd2 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:47.896926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 748/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:08:48.246047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:08:48.247296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:08:48.258858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe535de-6150-4b02-9335-a19f29eef713 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:08:48.357241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:08:48.357916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:08:48.358199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:08:48.358475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] PATH_INFO: `/v3/projects/3dcbf6efa92344d19e35a42b914d692c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:08:48.368337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dcbf6efa92344d19e35a42b914d692c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:08:48.369360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:08:48.369360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:08:48.369360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:08:48.370668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:08:48.429593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dcbf6efa92344d19e35a42b914d692c', 'request_id': 'req-835c8a02-8794-48d0-90ff-536570960fc7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:48.435972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dcbf6efa92344d19e35a42b914d692c', 'request_id': 'req-835c8a02-8794-48d0-90ff-536570960fc7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:48.449553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dcbf6efa92344d19e35a42b914d692c', 'request_id': 'req-835c8a02-8794-48d0-90ff-536570960fc7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:48.474743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] The token cache is being invalidate because project 3dcbf6efa92344d19e35a42b914d692c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:08:48.475028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:48.475538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-835c8a02-8794-48d0-90ff-536570960fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:08:48.476819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 762/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:08:48 2022] DELETE /identity/v3/projects/3dcbf6efa92344d19e35a42b914d692c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:08:57.894725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=62 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:08:58.472684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:10:12.706401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e7d03-3b1f-4636-8539-f1c57eac8368 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:12.707017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e7d03-3b1f-4636-8539-f1c57eac8368 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:12.707017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e7d03-3b1f-4636-8539-f1c57eac8368 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:12.708271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:12.713698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c75b89-b818-40d2-923a-fcda51403000 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:12.714139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c75b89-b818-40d2-923a-fcda51403000 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:12.714256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c75b89-b818-40d2-923a-fcda51403000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:12.792905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b9c75b89-b818-40d2-923a-fcda51403000 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:12.794057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b9c75b89-b818-40d2-923a-fcda51403000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:12.893806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:10:13.046815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e7d03-3b1f-4636-8539-f1c57eac8368 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:13.048811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ce2e7d03-3b1f-4636-8539-f1c57eac8368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:13.167247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e7d03-3b1f-4636-8539-f1c57eac8368 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:13.172760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d4a90-cd51-40e3-a53f-a2678508499f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:13.173400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4a90-cd51-40e3-a53f-a2678508499f service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:13.173679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4a90-cd51-40e3-a53f-a2678508499f service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:13.173983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4a90-cd51-40e3-a53f-a2678508499f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:13.183373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d4a90-cd51-40e3-a53f-a2678508499f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:13.184958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d4a90-cd51-40e3-a53f-a2678508499f service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:13.196112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 750/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:14.292866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e11ae-6354-4211-bb94-1b601d2f8ded None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:14.293482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e11ae-6354-4211-bb94-1b601d2f8ded None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:14.293482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e11ae-6354-4211-bb94-1b601d2f8ded None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:14.294910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:14.301324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb674909-6301-40b4-a7cb-73467245a50e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:14.301812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb674909-6301-40b4-a7cb-73467245a50e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:14.301917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb674909-6301-40b4-a7cb-73467245a50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:14.322005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-eb674909-6301-40b4-a7cb-73467245a50e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:14.323262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eb674909-6301-40b4-a7cb-73467245a50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:14.335612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:10:14.489279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e11ae-6354-4211-bb94-1b601d2f8ded None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:14.503418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e11ae-6354-4211-bb94-1b601d2f8ded None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:14.509486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a2ffc-c09e-4935-a569-b205454e5775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:14.510103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2ffc-c09e-4935-a569-b205454e5775 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:14.510389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2ffc-c09e-4935-a569-b205454e5775 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:14.510665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2ffc-c09e-4935-a569-b205454e5775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:14.518942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a2ffc-c09e-4935-a569-b205454e5775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:14.520520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a2ffc-c09e-4935-a569-b205454e5775 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:14.531474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 765/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:14.642886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cf9a4-f23b-4ee2-8c44-9d9b3bcbbc56 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:14.643097 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cf9a4-f23b-4ee2-8c44-9d9b3bcbbc56 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:14.643382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6cf9a4-f23b-4ee2-8c44-9d9b3bcbbc56 None None] PATH_INFO: `/v3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:14.644666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 752/1517] 10.222.0.23 () {54 vars in 925 bytes} [Tue Sep 13 14:10:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:10:14.676463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a06e37-73fb-49a0-842a-7b2bdd12a529 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:14.676744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a06e37-73fb-49a0-842a-7b2bdd12a529 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:14.677017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a06e37-73fb-49a0-842a-7b2bdd12a529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:14.736488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-62a06e37-73fb-49a0-842a-7b2bdd12a529 None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:14.737664 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-62a06e37-73fb-49a0-842a-7b2bdd12a529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:14.754376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:10:14.807574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 766/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 13 14:10:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:10:18.392182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea9609-b07b-46e6-9df9-c15ba7fb02b7 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:18.392693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea9609-b07b-46e6-9df9-c15ba7fb02b7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:18.392693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea9609-b07b-46e6-9df9-c15ba7fb02b7 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:18.394140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 753/1519] 10.222.0.20 () {54 vars in 961 bytes} [Tue Sep 13 14:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:18.399510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5715f-974d-41ec-86c8-f7e971e7ae26 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:18.399987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5715f-974d-41ec-86c8-f7e971e7ae26 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:18.400242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5715f-974d-41ec-86c8-f7e971e7ae26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:18.421451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0ab5715f-974d-41ec-86c8-f7e971e7ae26 None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:18.422701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0ab5715f-974d-41ec-86c8-f7e971e7ae26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:18.432039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 767/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Sep 13 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:10:18.439939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea9609-b07b-46e6-9df9-c15ba7fb02b7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:18.453700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea9609-b07b-46e6-9df9-c15ba7fb02b7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:18.458571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37154a-e046-4be8-8dba-16abf8ce0f3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:18.459151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37154a-e046-4be8-8dba-16abf8ce0f3b service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:18.459424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37154a-e046-4be8-8dba-16abf8ce0f3b service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:18.459690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37154a-e046-4be8-8dba-16abf8ce0f3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:18.467855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37154a-e046-4be8-8dba-16abf8ce0f3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:18.469396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37154a-e046-4be8-8dba-16abf8ce0f3b service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:18.478695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 754/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:25.998738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82749f-4fbd-44e2-87f2-49d2835c0c98 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:25.999313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82749f-4fbd-44e2-87f2-49d2835c0c98 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:25.999313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82749f-4fbd-44e2-87f2-49d2835c0c98 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:26.000650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:26.006793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebaff-317b-4850-81ab-15c62fdb9ab5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:26.007241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebaff-317b-4850-81ab-15c62fdb9ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:26.007357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebaff-317b-4850-81ab-15c62fdb9ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:26.027604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4aeebaff-317b-4850-81ab-15c62fdb9ab5 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:26.028797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4aeebaff-317b-4850-81ab-15c62fdb9ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:26.036565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:10:26.189904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82749f-4fbd-44e2-87f2-49d2835c0c98 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:26.205149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82749f-4fbd-44e2-87f2-49d2835c0c98 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:26.210817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa970b-c5bb-473b-ad5c-fac68ae23bec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:26.211622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa970b-c5bb-473b-ad5c-fac68ae23bec service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:26.211906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa970b-c5bb-473b-ad5c-fac68ae23bec service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:26.212177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa970b-c5bb-473b-ad5c-fac68ae23bec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:26.221459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa970b-c5bb-473b-ad5c-fac68ae23bec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:26.223076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa970b-c5bb-473b-ad5c-fac68ae23bec service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:26.234304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 769/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:27.357446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7461ac0-0028-421b-9ed3-eb6d8c2b157b None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:27.358033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7461ac0-0028-421b-9ed3-eb6d8c2b157b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:27.358033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7461ac0-0028-421b-9ed3-eb6d8c2b157b None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:27.359390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 14:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:27.365521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47679109-37f7-4e7e-8fdf-ec42ff06cc59 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:27.365955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47679109-37f7-4e7e-8fdf-ec42ff06cc59 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:27.366090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47679109-37f7-4e7e-8fdf-ec42ff06cc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:27.386153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-47679109-37f7-4e7e-8fdf-ec42ff06cc59 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:27.387344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47679109-37f7-4e7e-8fdf-ec42ff06cc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:27.395160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 14:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:10:27.548364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7461ac0-0028-421b-9ed3-eb6d8c2b157b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:27.563324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7461ac0-0028-421b-9ed3-eb6d8c2b157b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:27.568744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace4827-4765-40a0-9b77-9daff38154f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:27.569331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace4827-4765-40a0-9b77-9daff38154f9 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:27.569608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace4827-4765-40a0-9b77-9daff38154f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:27.569920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace4827-4765-40a0-9b77-9daff38154f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:27.578529 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace4827-4765-40a0-9b77-9daff38154f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:27.580083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace4827-4765-40a0-9b77-9daff38154f9 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:27.589413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 757/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:31.600452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ae582-bacf-4d8b-9d61-cd96ba08725c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:31.600989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ae582-bacf-4d8b-9d61-cd96ba08725c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:31.600989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ae582-bacf-4d8b-9d61-cd96ba08725c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:31.602355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:31.607677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c95af-0765-4bd2-947f-a177cbbdd9c7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:31.608110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c95af-0765-4bd2-947f-a177cbbdd9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:31.608216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c95af-0765-4bd2-947f-a177cbbdd9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:31.627589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-708c95af-0765-4bd2-947f-a177cbbdd9c7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:31.628771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-708c95af-0765-4bd2-947f-a177cbbdd9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:31.637275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:10:31.924296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ae582-bacf-4d8b-9d61-cd96ba08725c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:31.926102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa7ae582-bacf-4d8b-9d61-cd96ba08725c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:32.003250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ae582-bacf-4d8b-9d61-cd96ba08725c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:32.006842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98c0c8-b452-4785-bfc2-f078bd771654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:32.007440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98c0c8-b452-4785-bfc2-f078bd771654 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:32.007721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98c0c8-b452-4785-bfc2-f078bd771654 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:32.007999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98c0c8-b452-4785-bfc2-f078bd771654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:32.017511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98c0c8-b452-4785-bfc2-f078bd771654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:32.018995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98c0c8-b452-4785-bfc2-f078bd771654 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:32.028751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 772/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:34.159734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e8718-9dd4-4433-b9cf-c67b17a24ddf None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:34.160317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e8718-9dd4-4433-b9cf-c67b17a24ddf None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:34.160317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e8718-9dd4-4433-b9cf-c67b17a24ddf None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:34.161602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:34.166794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f20399-0e97-489d-a298-8bfda8489ac0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:34.167203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f20399-0e97-489d-a298-8bfda8489ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:34.167349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f20399-0e97-489d-a298-8bfda8489ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:34.186908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d8f20399-0e97-489d-a298-8bfda8489ac0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:34.188120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d8f20399-0e97-489d-a298-8bfda8489ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:34.199101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:10:34.499966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e8718-9dd4-4433-b9cf-c67b17a24ddf None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:34.511608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e8718-9dd4-4433-b9cf-c67b17a24ddf None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:34.515206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc0795-6db7-4af8-9334-34ec906e4851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:34.515793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc0795-6db7-4af8-9334-34ec906e4851 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:34.516074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc0795-6db7-4af8-9334-34ec906e4851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:34.516346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc0795-6db7-4af8-9334-34ec906e4851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:34.526508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc0795-6db7-4af8-9334-34ec906e4851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:34.527934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc0795-6db7-4af8-9334-34ec906e4851 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:34.537464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 760/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:38.164042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859f87f-bee4-4fdb-b56d-feb0e5675db8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:38.164590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859f87f-bee4-4fdb-b56d-feb0e5675db8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:38.164590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859f87f-bee4-4fdb-b56d-feb0e5675db8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:38.166502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:38.171804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783cd6-7a1e-4c81-b7a2-519b618e84b6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:38.172235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783cd6-7a1e-4c81-b7a2-519b618e84b6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:38.172331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b783cd6-7a1e-4c81-b7a2-519b618e84b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:38.179890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:10:38.193384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1b783cd6-7a1e-4c81-b7a2-519b618e84b6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:38.194617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1b783cd6-7a1e-4c81-b7a2-519b618e84b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:38.202513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:10:38.506375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859f87f-bee4-4fdb-b56d-feb0e5675db8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:38.508112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e859f87f-bee4-4fdb-b56d-feb0e5675db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:38.574414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:10:38.587853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859f87f-bee4-4fdb-b56d-feb0e5675db8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:38.591439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20af5361-e8a2-4ae9-8a03-ce8fd38687f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:38.592031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5361-e8a2-4ae9-8a03-ce8fd38687f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:38.592307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5361-e8a2-4ae9-8a03-ce8fd38687f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:38.592578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af5361-e8a2-4ae9-8a03-ce8fd38687f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:38.601260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af5361-e8a2-4ae9-8a03-ce8fd38687f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:38.602696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20af5361-e8a2-4ae9-8a03-ce8fd38687f1 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:38.611815 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 775/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:40.747032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498420af-8155-4548-91f7-63c65a098500 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:40.747587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498420af-8155-4548-91f7-63c65a098500 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:40.747587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498420af-8155-4548-91f7-63c65a098500 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:40.748964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 14:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:10:40.754394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d48c7-c771-49ca-85ff-810d3c82a8ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:40.754798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d48c7-c771-49ca-85ff-810d3c82a8ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:40.754944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d48c7-c771-49ca-85ff-810d3c82a8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:40.775766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-985d48c7-c771-49ca-85ff-810d3c82a8ff None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:10:40.776968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-985d48c7-c771-49ca-85ff-810d3c82a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:40.784772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 14:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:10:41.086581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498420af-8155-4548-91f7-63c65a098500 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:10:41.089578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-498420af-8155-4548-91f7-63c65a098500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:10:41.102323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498420af-8155-4548-91f7-63c65a098500 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:10:41.106444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588b6cf-f131-401a-bb8f-d3785f523baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:10:41.107030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588b6cf-f131-401a-bb8f-d3785f523baa service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:10:41.107308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588b6cf-f131-401a-bb8f-d3785f523baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:10:41.107582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588b6cf-f131-401a-bb8f-d3785f523baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:10:41.116351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588b6cf-f131-401a-bb8f-d3785f523baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:10:41.117779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588b6cf-f131-401a-bb8f-d3785f523baa service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:10:41.127415 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 763/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:10:50.770030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=11 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:10:51.122963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=8 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:06.259941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f2af-8eab-43e1-8e2e-a9fba73e571a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.260577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f2af-8eab-43e1-8e2e-a9fba73e571a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.260577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f2af-8eab-43e1-8e2e-a9fba73e571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.285978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6681f2af-8eab-43e1-8e2e-a9fba73e571a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:06.287189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6681f2af-8eab-43e1-8e2e-a9fba73e571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:06.300064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 777/1540] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:06.310364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297bcfd-758c-442d-a42f-7d14505bb779 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.311416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297bcfd-758c-442d-a42f-7d14505bb779 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.311916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297bcfd-758c-442d-a42f-7d14505bb779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.338139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6297bcfd-758c-442d-a42f-7d14505bb779 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:06.340068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6297bcfd-758c-442d-a42f-7d14505bb779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:06.353657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 764/1541] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:06.358300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681f2af-8eab-43e1-8e2e-a9fba73e571a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:06.373364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681f2af-8eab-43e1-8e2e-a9fba73e571a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:06.381622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:06.382289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.382567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.382843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.383516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:06.383833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:06.386081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:06.415950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 778/1542] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:06.424807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297bcfd-758c-442d-a42f-7d14505bb779 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:06.435988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297bcfd-758c-442d-a42f-7d14505bb779 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:06.445824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:06.446471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.446754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.447029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.447696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:06.449692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:06.519507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 765/1543] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:06.539115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:06.555569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512baf4-4dd0-4ee4-959c-3bb29cf57e4b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:06.637884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:06.638616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.638901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.639176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.639817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:06.641939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:06.671833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 779/1544] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:06.676110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:06.689583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7cc86-a082-4d92-9728-55711cb90965 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:06.697780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573acfa5-a1dd-4212-b674-3e1c9f12c15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:06.698439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573acfa5-a1dd-4212-b674-3e1c9f12c15a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.698723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573acfa5-a1dd-4212-b674-3e1c9f12c15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.699001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573acfa5-a1dd-4212-b674-3e1c9f12c15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.699718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573acfa5-a1dd-4212-b674-3e1c9f12c15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:06.701336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573acfa5-a1dd-4212-b674-3e1c9f12c15a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:06.717316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 766/1545] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:06.720983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:06.732923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb59aff-4bd6-4e47-8362-48020c05c99c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.733246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb59aff-4bd6-4e47-8362-48020c05c99c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.733620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb59aff-4bd6-4e47-8362-48020c05c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.736154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c412-5b87-48a7-961e-bb8ad3af4777 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:06.744495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab1c00-16b0-45d7-ad87-c75195576042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:06.745128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab1c00-16b0-45d7-ad87-c75195576042 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.745405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab1c00-16b0-45d7-ad87-c75195576042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.745676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab1c00-16b0-45d7-ad87-c75195576042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.747882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab1c00-16b0-45d7-ad87-c75195576042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:06.749606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab1c00-16b0-45d7-ad87-c75195576042 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:06.774105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 780/1546] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:06.776234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232c31-2c82-409b-bef2-775f3ffd82ce None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.776515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232c31-2c82-409b-bef2-775f3ffd82ce None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.776788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232c31-2c82-409b-bef2-775f3ffd82ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.778738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4eb59aff-4bd6-4e47-8362-48020c05c99c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:06.780000 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4eb59aff-4bd6-4e47-8362-48020c05c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:06.802965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 767/1547] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:06.804004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb59aff-4bd6-4e47-8362-48020c05c99c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:06.830853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-25232c31-2c82-409b-bef2-775f3ffd82ce None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:06.832052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-25232c31-2c82-409b-bef2-775f3ffd82ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:06.835686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb59aff-4bd6-4e47-8362-48020c05c99c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:06.846217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448c65c-c8b8-4bda-81b3-7e0ed41caadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:06.848650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448c65c-c8b8-4bda-81b3-7e0ed41caadb admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.849023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448c65c-c8b8-4bda-81b3-7e0ed41caadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.849441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448c65c-c8b8-4bda-81b3-7e0ed41caadb admin admin] PATH_INFO: `/v3/projects/e206bc45915744e29dcb918218fa3416/users/92c30fba107b4dfbb7b7b1582cee2ae8/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.855142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 781/1548] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:06.857438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47127eb5-664c-4735-9377-80fab0631476 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.857722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47127eb5-664c-4735-9377-80fab0631476 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.858020 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47127eb5-664c-4735-9377-80fab0631476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.884207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448c65c-c8b8-4bda-81b3-7e0ed41caadb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e206bc45915744e29dcb918218fa3416, user_id=92c30fba107b4dfbb7b7b1582cee2ae8, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:06.886901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448c65c-c8b8-4bda-81b3-7e0ed41caadb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:06.903727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-47127eb5-664c-4735-9377-80fab0631476 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:06.905391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47127eb5-664c-4735-9377-80fab0631476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:06.919465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 768/1549] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:06 2022] PUT /identity/v3/projects/e206bc45915744e29dcb918218fa3416/users/92c30fba107b4dfbb7b7b1582cee2ae8/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:06.923314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002725f0-763f-4c1c-a5a7-71d34bc6c615 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:06.923635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002725f0-763f-4c1c-a5a7-71d34bc6c615 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:06.923863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002725f0-763f-4c1c-a5a7-71d34bc6c615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:06.974324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-002725f0-763f-4c1c-a5a7-71d34bc6c615 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:06.976470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-002725f0-763f-4c1c-a5a7-71d34bc6c615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.032734 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 782/1550] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.039306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48fdcb-7c0c-4ee7-8aee-216c766bd5f4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.039643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48fdcb-7c0c-4ee7-8aee-216c766bd5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.039928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48fdcb-7c0c-4ee7-8aee-216c766bd5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.052749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 769/1551] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.054828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcee3a-8c49-4f6d-b568-b614f7aa1a7d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.055283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcee3a-8c49-4f6d-b568-b614f7aa1a7d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.056356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcee3a-8c49-4f6d-b568-b614f7aa1a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.064314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ab48fdcb-7c0c-4ee7-8aee-216c766bd5f4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.066942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab48fdcb-7c0c-4ee7-8aee-216c766bd5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.076639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 783/1552] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.078040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec9fd-20f9-4461-ad97-2b80b9a67855 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.078326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec9fd-20f9-4461-ad97-2b80b9a67855 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.078600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cec9fd-20f9-4461-ad97-2b80b9a67855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.083656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a0bcee3a-8c49-4f6d-b568-b614f7aa1a7d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.085555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a0bcee3a-8c49-4f6d-b568-b614f7aa1a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.094677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 770/1553] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.096228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d02f33-f8e9-48cb-997d-1667e2e325f4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.097161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d02f33-f8e9-48cb-997d-1667e2e325f4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.097609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d02f33-f8e9-48cb-997d-1667e2e325f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.101178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-21cec9fd-20f9-4461-ad97-2b80b9a67855 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.102485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-21cec9fd-20f9-4461-ad97-2b80b9a67855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.116747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 784/1554] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.116747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cec9fd-20f9-4461-ad97-2b80b9a67855 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.123884 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cec9fd-20f9-4461-ad97-2b80b9a67855 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.127813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d4d02f33-f8e9-48cb-997d-1667e2e325f4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.129484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d4d02f33-f8e9-48cb-997d-1667e2e325f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.132716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.133429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.133713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.134018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.134680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:07.134986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.136686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.138475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 771/1555] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.139630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976fb31-c023-4050-a785-a81fccc55dba None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.139906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976fb31-c023-4050-a785-a81fccc55dba None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.140172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976fb31-c023-4050-a785-a81fccc55dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.150290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 785/1556] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:07.151295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.161255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7976fb31-c023-4050-a785-a81fccc55dba None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.161822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d698b-f47b-4642-aa34-a5d8149927a1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.162460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7976fb31-c023-4050-a785-a81fccc55dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.167139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.167738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.168014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.168283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.168922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:07.169224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.170866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.171206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 772/1557] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.172468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcec11-7345-463d-9b37-cbcd56294a95 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.172755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcec11-7345-463d-9b37-cbcd56294a95 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.173027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcec11-7345-463d-9b37-cbcd56294a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.184909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 786/1558] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:07.185716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.198632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-02dcec11-7345-463d-9b37-cbcd56294a95 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.200103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-02dcec11-7345-463d-9b37-cbcd56294a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.201933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:07.210020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 773/1559] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.211475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e9a6a-cfef-4dba-a33e-bc1b460be633 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.211754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e9a6a-cfef-4dba-a33e-bc1b460be633 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.212026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e9a6a-cfef-4dba-a33e-bc1b460be633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.234634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e17e9a6a-cfef-4dba-a33e-bc1b460be633 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.235839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e17e9a6a-cfef-4dba-a33e-bc1b460be633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.245011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 774/1560] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.249745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c8d88-0626-4816-b2e8-524f900d1a94 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.250031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c8d88-0626-4816-b2e8-524f900d1a94 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.250305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c8d88-0626-4816-b2e8-524f900d1a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.252837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be76ddb-b79a-47d1-a68c-f980a03b0add admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.257467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.259838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.260121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.260647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.269803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.271278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.271477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-080c8d88-0626-4816-b2e8-524f900d1a94 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.272958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-080c8d88-0626-4816-b2e8-524f900d1a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.280907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 775/1561] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.281280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 787/1562] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:07.281918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bd95b-f30e-43f5-b19d-8beedbe50bda None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.282046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.282201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bd95b-f30e-43f5-b19d-8beedbe50bda None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.282475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bd95b-f30e-43f5-b19d-8beedbe50bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.293354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059be9eb-8793-459e-b48e-bdbf362ba377 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.298535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.299138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.299421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.299797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.300415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.301436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:12:07.301436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:12:07.301436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:12:07.301761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ca3bd95b-f30e-43f5-b19d-8beedbe50bda None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.302659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.302787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ca3bd95b-f30e-43f5-b19d-8beedbe50bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.311680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 776/1563] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.312750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bd95b-f30e-43f5-b19d-8beedbe50bda None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.323924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bd95b-f30e-43f5-b19d-8beedbe50bda None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.329074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.330496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.331051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.331441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.332314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:07.332714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.333866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 788/1564] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:07.334020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:12:07.334020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Sep 13 14:12:07.334020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: Sep 13 14:12:07.334915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.335203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754104ad-41b1-4cb2-8110-9ef7317903fd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.348402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 777/1565] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:07.349528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4e7d-a90c-4cc2-b958-c95ebae5d783 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.349965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4e7d-a90c-4cc2-b958-c95ebae5d783 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.350698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4e7d-a90c-4cc2-b958-c95ebae5d783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.373505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b13f4e7d-a90c-4cc2-b958-c95ebae5d783 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.374958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b13f4e7d-a90c-4cc2-b958-c95ebae5d783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.394551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:07.416736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01eb1af-156b-4f39-84e8-39e17eb0c581 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.423801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d59fe-687e-4e7e-bd9f-b886297a98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.424424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d59fe-687e-4e7e-bd9f-b886297a98b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.424705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d59fe-687e-4e7e-bd9f-b886297a98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.424979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d59fe-687e-4e7e-bd9f-b886297a98b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.425646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d59fe-687e-4e7e-bd9f-b886297a98b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.427760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d59fe-687e-4e7e-bd9f-b886297a98b9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.452328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 778/1566] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.452328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4e7d-a90c-4cc2-b958-c95ebae5d783 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.463943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 789/1567] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:07.465892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d065253-508b-4719-ab14-b8ba32d26514 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.466184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d065253-508b-4719-ab14-b8ba32d26514 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.466526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d065253-508b-4719-ab14-b8ba32d26514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.467865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4e7d-a90c-4cc2-b958-c95ebae5d783 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.493261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1d065253-508b-4719-ab14-b8ba32d26514 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:07.494662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1d065253-508b-4719-ab14-b8ba32d26514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:07.546801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.547339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.547620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.547891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.549232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:07.549541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.551207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.565782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 779/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:07.565782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.576489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8cea8-cc11-4e5b-907c-dea9a0c16d83 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.581480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.582184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.582540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.582888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.583612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:07.584032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.585703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.591853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 790/1569] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:07.591853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d065253-508b-4719-ab14-b8ba32d26514 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.603258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d065253-508b-4719-ab14-b8ba32d26514 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.603976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 780/1570] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:07.604887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.615308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.616623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7e4cf-673c-4e07-ad8f-c27bfe91b0da admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.616904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.617411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.618046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.619065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.621088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.624520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.625484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.625895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.626817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.627706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.631246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.644822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 791/1571] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/users => generated 290 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:07.645954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.653315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:07.661420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3c771-d312-4771-9ccf-eb75b59c4948 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.667832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 781/1572] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:07.667832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.741199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.741868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.742148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.742473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.743136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:07.743439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.745595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.757509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e344ed-56cd-45b4-98f6-00b952778c34 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.763145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.764226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.764798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.765244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.766091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.767471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:12:07.767471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Sep 13 14:12:07.767471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: Sep 13 14:12:07.769083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.770974 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 792/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:07.773301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.785837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b4ede-5c53-4780-a387-f6a48a64f6ae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.793117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.793765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.794138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.794417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.795256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.797859 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.799491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 782/1574] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/users => generated 285 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:07.800848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.812771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cafff-a9ab-448d-adaf-70c4a852df2f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.818481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.820102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.820467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.820854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.821750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:07.822233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.823943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.832897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:07.837706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 793/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/projects => generated 389 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:07.838789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.852290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 783/1576] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:07.852290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.859196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd395658-2bdb-420b-86d6-13d4b37c4063 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.941600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.942245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.942524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.942795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.942925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a7ced-ffa0-47ed-8b84-b403c8902763 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.943498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.946246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.949953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.951342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.951634 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.951911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.952606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.954729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.959301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 784/1577] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:07.959301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:07.972029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a0a4f-7ddd-4920-b783-9167026123c4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:07.978455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:07.980391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:07.982263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:07.982610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:07.983264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:07.985780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:07.989251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 794/1578] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:07.990178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.012379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 785/1579] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/users => generated 292 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:08.013388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.026377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f985-7cd7-4d5e-9367-75cee3476744 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.072299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a250e0a-91ee-4211-ba44-2ff97477824a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.077410 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.078080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.078355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.078621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.079471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.081675 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.087876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:08.098345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.099349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.099703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.100366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.101329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.103168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.119946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 795/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:07 2022] POST /identity/v3/projects => generated 357 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:08.120160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 786/1581] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.120160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.120323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.130077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9651e-4803-418b-9ef4-9c5d4c02691f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.194808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6490b3a5-d9a5-42ad-af88-f5ada1e8c8f9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.200836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.201593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.202728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.202728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.203065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.204873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.213753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.214387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.214690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.215592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.216271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.218490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.231775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 796/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:08 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:08.232721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.244153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1f528-340b-4d68-92fd-2baeb0eb73ce admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.251787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.252559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.252878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.253157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.253648 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:08.254025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.255784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.270943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 787/1583] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:08 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:08.271372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.276800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 797/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.277500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.295329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e38a8b-7a93-40c5-b801-b2eb94d4f50a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.356918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd414e36-5605-4585-8cc5-6af085ffb517 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.362685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.363304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.363778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.364057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.364741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.370831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.373507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.374151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.374429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.374749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.375398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.376987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.389784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 798/1585] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.389784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.404811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd952-4fa3-4e6b-9c45-8d201262a972 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.411521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 788/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:08 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:08.411521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.411783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.411783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.411783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.412060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.413438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.415161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.423787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7d8d7-4e71-4751-9953-c21c54f614a7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.427355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 799/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.429029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.429985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.430899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.431119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.431353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.431994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.434131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.444079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647251da-9b00-4cfa-9dad-56a9123996f9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.450355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.451787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.452241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.452842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.453797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.455876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.461428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 789/1588] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:08 2022] POST /identity/v3/users => generated 279 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:08.462476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.469914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 800/1589] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.471155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.476098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874140b2-7983-46a0-b191-ea6ecb408a00 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.481426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.482095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.482379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.482652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.483447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.485194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50c416-5caf-4995-9f5f-4eda8c90ff1e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.486630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.491529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.493379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.493902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.494495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] PATH_INFO: `/v3/projects/a66ebe0eb0ec44fa8fb3ed62d27f9599/users/e35dfe8442964bd793276386c2fe6f95/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.514603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66ebe0eb0ec44fa8fb3ed62d27f9599, user_id=e35dfe8442964bd793276386c2fe6f95, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.516380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:12:08.516380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:12:08.516380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:12:08.518878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.524302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 790/1590] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:08 2022] POST /identity/v3/users => generated 284 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:08.525282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.544068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9cbf6-d297-49d4-8747-52b4eddb356b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.547791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 801/1591] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:08 2022] PUT /identity/v3/projects/a66ebe0eb0ec44fa8fb3ed62d27f9599/users/e35dfe8442964bd793276386c2fe6f95/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:08.552365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.570223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:08.618151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.618671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.618950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.619222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.620125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.622766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.637511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 791/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.641701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.642183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1eee1-2ee3-4860-aabf-9976d47f550c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.647291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.648069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.648352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.649059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.649259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.650897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.654384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf36444-23c3-410e-8e06-c930dc207f0f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.660646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.661414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.664835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.664980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 802/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.665082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] PATH_INFO: `/v3/projects/5a575c06adfd4cb8a6722de49994004c/users/d35f22e67dbc4f9e9144596860b21e79/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.665210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.673154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343493c-a689-4368-b0b1-cc2c39256bd8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.677868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.678979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.679256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.679525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.680582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.681774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.683162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a575c06adfd4cb8a6722de49994004c, user_id=d35f22e67dbc4f9e9144596860b21e79, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.686180 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.693643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 803/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.697983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.713772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 792/1595] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:08 2022] PUT /identity/v3/projects/5a575c06adfd4cb8a6722de49994004c/users/d35f22e67dbc4f9e9144596860b21e79/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:08.713772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.719578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a7e5e-b8b4-493e-9ea1-6d0986d115d0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.733839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:08.778639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd64b5-b2b4-45cb-95da-214cb3d96f1b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.784665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5dcd2-b77e-45ca-bd50-63fb8151e1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.785280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5dcd2-b77e-45ca-bd50-63fb8151e1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.785556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5dcd2-b77e-45ca-bd50-63fb8151e1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.785856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5dcd2-b77e-45ca-bd50-63fb8151e1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.786499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5dcd2-b77e-45ca-bd50-63fb8151e1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.789395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5dcd2-b77e-45ca-bd50-63fb8151e1e6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.801931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 793/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.803330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ecafa-e3eb-4fc7-9cbb-88bb596aa652 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.803621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ecafa-e3eb-4fc7-9cbb-88bb596aa652 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.803961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ecafa-e3eb-4fc7-9cbb-88bb596aa652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.826481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-dd6ecafa-e3eb-4fc7-9cbb-88bb596aa652 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:08.827428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.827707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dd6ecafa-e3eb-4fc7-9cbb-88bb596aa652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:08.828245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.828735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.829086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.829906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.831939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.837926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 794/1597] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:08.842828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ecafa-e3eb-4fc7-9cbb-88bb596aa652 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.851772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 804/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.851772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.859539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ecafa-e3eb-4fc7-9cbb-88bb596aa652 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.863560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40fe77-9070-440f-8d31-9f32227d94c2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.868221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f55e1-bdac-4b7b-8d8d-211e0346950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.868893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f55e1-bdac-4b7b-8d8d-211e0346950f admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.869250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f55e1-bdac-4b7b-8d8d-211e0346950f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.870464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f55e1-bdac-4b7b-8d8d-211e0346950f admin admin] PATH_INFO: `/v3/projects/6136ce7badef4cda8912aa34203a2cc0/users/a22a06667e10411e8d8ce92db4536a1a/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.871132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.872321 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.872786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.873751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.874740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.876567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.893148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f55e1-bdac-4b7b-8d8d-211e0346950f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6136ce7badef4cda8912aa34203a2cc0, user_id=a22a06667e10411e8d8ce92db4536a1a, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:08.899337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 805/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:08.899728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:08.903532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f55e1-bdac-4b7b-8d8d-211e0346950f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:08.913759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d3279-2b1f-488f-be52-b82300806c13 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:08.930149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 795/1600] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:08 2022] PUT /identity/v3/projects/6136ce7badef4cda8912aa34203a2cc0/users/a22a06667e10411e8d8ce92db4536a1a/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:08.930149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8f76f-30bb-4ab8-8270-f3c93acb3c5c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:08.930149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8f76f-30bb-4ab8-8270-f3c93acb3c5c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:08.930149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8f76f-30bb-4ab8-8270-f3c93acb3c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:08.968922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f5e8f76f-30bb-4ab8-8270-f3c93acb3c5c None None] MFA Rules not processed for user `d35f22e67dbc4f9e9144596860b21e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:08.970551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f5e8f76f-30bb-4ab8-8270-f3c93acb3c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:08.999177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:08.999953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.000858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.001278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.001963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.004190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.017232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 806/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:09.018684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.032221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419224cc-be08-45b6-aefc-0b3756964a9d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.038849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.039493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.039883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.040164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.040886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.042572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.061769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 807/1602] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:09.061769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.073479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:09.074401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 796/1603] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:09.075340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8f76f-30bb-4ab8-8270-f3c93acb3c5c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.092074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8f76f-30bb-4ab8-8270-f3c93acb3c5c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.101419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.102144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.102427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.102704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] PATH_INFO: `/v3/projects/4a11c6709cba4247acc57bd97250f8a0/users/961012089bd147ecab4fcdf5ea3ae4c3/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.135293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a11c6709cba4247acc57bd97250f8a0, user_id=961012089bd147ecab4fcdf5ea3ae4c3, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.138424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.151756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a875f4-6a42-4267-8f04-059d1599e8e6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.157445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 797/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:09 2022] PUT /identity/v3/projects/4a11c6709cba4247acc57bd97250f8a0/users/961012089bd147ecab4fcdf5ea3ae4c3/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:09.161820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.217866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.218552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.219526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.219924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.233935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:09.241187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192672-d954-4847-a1c5-16eecdc3b89c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.252130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9b025-3858-4b4e-9a5a-484b71f27418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.253139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9b025-3858-4b4e-9a5a-484b71f27418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.253516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9b025-3858-4b4e-9a5a-484b71f27418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.254399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9b025-3858-4b4e-9a5a-484b71f27418 admin admin] PATH_INFO: `/v3/projects/2fd378f8a7b34d8d82f6a145c62862ac/users/17e8dbc75366467f8fc6f0681e722751/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.271417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9b025-3858-4b4e-9a5a-484b71f27418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd378f8a7b34d8d82f6a145c62862ac, user_id=17e8dbc75366467f8fc6f0681e722751, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.276889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9b025-3858-4b4e-9a5a-484b71f27418 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.297815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 798/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:09 2022] PUT /identity/v3/projects/2fd378f8a7b34d8d82f6a145c62862ac/users/17e8dbc75366467f8fc6f0681e722751/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:09.299448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29be91-72f4-4a11-aea5-8f20b153c5dc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.299961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29be91-72f4-4a11-aea5-8f20b153c5dc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.300445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29be91-72f4-4a11-aea5-8f20b153c5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.318378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.320189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.327748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cc29be91-72f4-4a11-aea5-8f20b153c5dc None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:09.328940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cc29be91-72f4-4a11-aea5-8f20b153c5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:09.335151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 808/1606] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:09.335947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.456682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733e121-9981-48ae-a325-95ad777f92b2 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.456682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440a08e-b41d-4aad-9a7a-82d28220a63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.456682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440a08e-b41d-4aad-9a7a-82d28220a63d admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.456682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440a08e-b41d-4aad-9a7a-82d28220a63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.457150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440a08e-b41d-4aad-9a7a-82d28220a63d admin admin] PATH_INFO: `/v3/projects/ac73625cb9244518a2165a776a9819d5/users/a2713c44481840d290cfda681adb9d01/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.472226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440a08e-b41d-4aad-9a7a-82d28220a63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac73625cb9244518a2165a776a9819d5, user_id=a2713c44481840d290cfda681adb9d01, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.475240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440a08e-b41d-4aad-9a7a-82d28220a63d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.477298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 799/1607] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:09.478483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29be91-72f4-4a11-aea5-8f20b153c5dc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.488679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29be91-72f4-4a11-aea5-8f20b153c5dc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.489852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:09.500009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 809/1608] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:09 2022] PUT /identity/v3/projects/ac73625cb9244518a2165a776a9819d5/users/a2713c44481840d290cfda681adb9d01/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:09.502222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa6168-9458-47ec-aaad-7d127ff7da51 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.502611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa6168-9458-47ec-aaad-7d127ff7da51 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.503052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa6168-9458-47ec-aaad-7d127ff7da51 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.504628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 810/1609] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 13 14:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:09.505898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ece3-bb02-47d3-b4e5-223d78cf81c1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.506536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ece3-bb02-47d3-b4e5-223d78cf81c1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.506949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45ece3-bb02-47d3-b4e5-223d78cf81c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.531651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dc45ece3-bb02-47d3-b4e5-223d78cf81c1 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:09.533862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dc45ece3-bb02-47d3-b4e5-223d78cf81c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:09.594076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.594789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.595483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.595771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] PATH_INFO: `/v3/projects/88abd18fe3a84f619312d663b59da63f/users/c90218bf52dd428c979cba8eda045eb2/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.613868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88abd18fe3a84f619312d663b59da63f, user_id=c90218bf52dd428c979cba8eda045eb2, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.615418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 811/1610] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:09.617198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.617505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec8bca-26ff-46bf-94ac-db46eaecd251 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.617853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec8bca-26ff-46bf-94ac-db46eaecd251 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.618095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec8bca-26ff-46bf-94ac-db46eaecd251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.634139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 800/1611] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:09 2022] PUT /identity/v3/projects/88abd18fe3a84f619312d663b59da63f/users/c90218bf52dd428c979cba8eda045eb2/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:09.635896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.637567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:09.641206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2bec8bca-26ff-46bf-94ac-db46eaecd251 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:09.642573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2bec8bca-26ff-46bf-94ac-db46eaecd251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:09.698080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c246848-b784-4fc2-a494-8f33db8b3f83 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.702649 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7e91f-c41e-4470-8731-b2ea3cbe72f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.703330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e91f-c41e-4470-8731-b2ea3cbe72f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.703573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e91f-c41e-4470-8731-b2ea3cbe72f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.703773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e91f-c41e-4470-8731-b2ea3cbe72f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.709755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 812/1612] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:09.714309 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181debb9-da34-4445-9620-c1025f9e9b60 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.714667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181debb9-da34-4445-9620-c1025f9e9b60 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.714915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181debb9-da34-4445-9620-c1025f9e9b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.720999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7e91f-c41e-4470-8731-b2ea3cbe72f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.722431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7e91f-c41e-4470-8731-b2ea3cbe72f4 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.734923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 801/1613] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:09.736553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f98cb4-7072-4b32-a35e-3113fed54416 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.736840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f98cb4-7072-4b32-a35e-3113fed54416 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.737115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f98cb4-7072-4b32-a35e-3113fed54416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.749767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:09.757329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-181debb9-da34-4445-9620-c1025f9e9b60 None None] MFA Rules not processed for user `a2713c44481840d290cfda681adb9d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:09.776244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-181debb9-da34-4445-9620-c1025f9e9b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:09.801387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-20f98cb4-7072-4b32-a35e-3113fed54416 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:09.803160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-20f98cb4-7072-4b32-a35e-3113fed54416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:09.891192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 813/1614] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:09.892197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181debb9-da34-4445-9620-c1025f9e9b60 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.898787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 802/1615] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 13 14:12:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:09.901246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512bba9-44a5-43e3-9e5b-5a1b269c1435 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.901546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512bba9-44a5-43e3-9e5b-5a1b269c1435 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.901848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512bba9-44a5-43e3-9e5b-5a1b269c1435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.905682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181debb9-da34-4445-9620-c1025f9e9b60 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.910785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.911543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.911913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.912323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:09.930793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:09.933291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:09.939589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7512bba9-44a5-43e3-9e5b-5a1b269c1435 None None] MFA Rules not processed for user `c90218bf52dd428c979cba8eda045eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:09.941476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7512bba9-44a5-43e3-9e5b-5a1b269c1435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:09.961371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 814/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:09.962872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:09.976973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae63e49-a0c3-4822-88dc-beedf5e4354d service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:09.982829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:09.983672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:09.984074 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:09.984496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.000170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.002481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.023751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 815/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:10.025118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.032297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:10.040838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1adb-bdd5-44c7-9e6a-85097cd0477e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.053703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.054241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 803/1618] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:10.055487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512bba9-44a5-43e3-9e5b-5a1b269c1435 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.070555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.070882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.072217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512bba9-44a5-43e3-9e5b-5a1b269c1435 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.075732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.076687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.079392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.079766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.080025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.080374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.080653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.124247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 816/1619] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:10 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:10.125493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.180271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.182123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.198846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 804/1620] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:10.199983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.219225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8695-d23e-4cbd-9724-1e12120a1c00 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.229052 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:10.229319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3780-9936-4624-8d26-d7f626c788f2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.240206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.242868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.244235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.244380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.331764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.332640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.332996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.333277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.334000 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.337288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.375512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 805/1621] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:10 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:10.378990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.403449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.405517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf548b6d-01b7-4fb2-9e8d-43be15d0ef02 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.407376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.431208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.433224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.435332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.436222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.436965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 817/1622] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:10.452934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.464429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.514329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 806/1623] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:12:10.517067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.543130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60da6c6-973f-41a7-bcae-f427846ed26c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.565500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19729b9d-8930-4e89-bcad-03932ee37c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.566607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729b9d-8930-4e89-bcad-03932ee37c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.567328 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729b9d-8930-4e89-bcad-03932ee37c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.567830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729b9d-8930-4e89-bcad-03932ee37c5f admin admin] PATH_INFO: `/v3/projects/03e4a7ff8c8d4f63a07d13f565fe3226/users/b05839c3a78b4593849b360f3573bee1/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.605327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19729b9d-8930-4e89-bcad-03932ee37c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e4a7ff8c8d4f63a07d13f565fe3226, user_id=b05839c3a78b4593849b360f3573bee1, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.608488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19729b9d-8930-4e89-bcad-03932ee37c5f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.646553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 818/1624] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:10 2022] PUT /identity/v3/projects/03e4a7ff8c8d4f63a07d13f565fe3226/users/b05839c3a78b4593849b360f3573bee1/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:10.651949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.663844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b9e3a-cf6a-4e56-bdca-453c52ea4f94 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.664222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b9e3a-cf6a-4e56-bdca-453c52ea4f94 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.664707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b9e3a-cf6a-4e56-bdca-453c52ea4f94 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.666572 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f8d09-cf1c-4127-9caf-18d11ef7017c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.666828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 819/1625] 10.222.0.20 () {54 vars in 912 bytes} [Tue Sep 13 14:12:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:10.672397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d192ba-4ada-4c81-8d34-1910388aa64b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.672812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d192ba-4ada-4c81-8d34-1910388aa64b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.673194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d192ba-4ada-4c81-8d34-1910388aa64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.707006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-92d192ba-4ada-4c81-8d34-1910388aa64b None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:10.712868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-92d192ba-4ada-4c81-8d34-1910388aa64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:10.796225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.797069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.798129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.798506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.800652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.805351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.830162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 807/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:10.833359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.857670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d82589-f613-41fc-ae86-5df3621d2842 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:10.868443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e647f2-73e5-4a62-83fc-9bbf21a24fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:10.869927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e647f2-73e5-4a62-83fc-9bbf21a24fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.871088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e647f2-73e5-4a62-83fc-9bbf21a24fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.878675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e647f2-73e5-4a62-83fc-9bbf21a24fa6 admin admin] PATH_INFO: `/v3/projects/03e4a7ff8c8d4f63a07d13f565fe3226/users/b05839c3a78b4593849b360f3573bee1/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.879281 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 820/1627] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:12:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:10.882251 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e647f2-73e5-4a62-83fc-9bbf21a24fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e4a7ff8c8d4f63a07d13f565fe3226, user_id=b05839c3a78b4593849b360f3573bee1, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:10.886760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d192ba-4ada-4c81-8d34-1910388aa64b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:10.889331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e647f2-73e5-4a62-83fc-9bbf21a24fa6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:10.893869 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:10.923691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 808/1628] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:10 2022] PUT /identity/v3/projects/03e4a7ff8c8d4f63a07d13f565fe3226/users/b05839c3a78b4593849b360f3573bee1/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:10.935513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf841ba-8d07-468c-95c7-d569384cd853 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:10.936118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf841ba-8d07-468c-95c7-d569384cd853 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:10.936577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf841ba-8d07-468c-95c7-d569384cd853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:10.995193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1cf841ba-8d07-468c-95c7-d569384cd853 None None] MFA Rules not processed for user `b05839c3a78b4593849b360f3573bee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:10.996572 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1cf841ba-8d07-468c-95c7-d569384cd853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:11.021597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d192ba-4ada-4c81-8d34-1910388aa64b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:11.095892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:11.097078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:11.097562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:11.098597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:11.134128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:11.151232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 809/1629] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:11.259221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:11.262817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:11.296211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 821/1630] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:11.599164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf841ba-8d07-468c-95c7-d569384cd853 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:11.614557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf841ba-8d07-468c-95c7-d569384cd853 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:11.627434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2183fd-8b77-4f99-9f68-09f397028d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:11.628904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2183fd-8b77-4f99-9f68-09f397028d97 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:11.632114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2183fd-8b77-4f99-9f68-09f397028d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:11.632539 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2183fd-8b77-4f99-9f68-09f397028d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:11.670296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2183fd-8b77-4f99-9f68-09f397028d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:11.671760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2183fd-8b77-4f99-9f68-09f397028d97 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:11.695383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 810/1631] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:12.113196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:12.122916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58477a5b-17d9-4124-acc5-a9474e7f68e4 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:12.136016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c8c27-3e76-4e38-84f1-77b25a1f0679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:12.136802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c8c27-3e76-4e38-84f1-77b25a1f0679 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:12.137763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c8c27-3e76-4e38-84f1-77b25a1f0679 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:12.137763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c8c27-3e76-4e38-84f1-77b25a1f0679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:12.306821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c8c27-3e76-4e38-84f1-77b25a1f0679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:12.310585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c8c27-3e76-4e38-84f1-77b25a1f0679 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:12.320032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:12.335245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 822/1632] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:12.905514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7329f8-f827-4ca8-8a73-864c4b5e81f3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:12.906098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7329f8-f827-4ca8-8a73-864c4b5e81f3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:12.906283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7329f8-f827-4ca8-8a73-864c4b5e81f3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:12.907903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 811/1633] 10.222.0.20 () {54 vars in 962 bytes} [Tue Sep 13 14:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:12.926335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aae15-d40c-4026-a8fd-7c386e2afa97 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:12.926619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aae15-d40c-4026-a8fd-7c386e2afa97 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:12.926898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35aae15-d40c-4026-a8fd-7c386e2afa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:12.967481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f35aae15-d40c-4026-a8fd-7c386e2afa97 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:12.968689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f35aae15-d40c-4026-a8fd-7c386e2afa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:12.981332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 823/1634] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Sep 13 14:12:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:12.986992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1b2c6-4edf-4b8b-ae1a-3b20fea806c0 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:12.987278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1b2c6-4edf-4b8b-ae1a-3b20fea806c0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:12.987558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1b2c6-4edf-4b8b-ae1a-3b20fea806c0 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:12.988876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 812/1635] 10.222.0.20 () {54 vars in 912 bytes} [Tue Sep 13 14:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:12.994773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbff6-91ae-4bae-86c9-b520d231c4e4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:12.995054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbff6-91ae-4bae-86c9-b520d231c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:12.995332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecbff6-91ae-4bae-86c9-b520d231c4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:13.050116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6fecbff6-91ae-4bae-86c9-b520d231c4e4 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:13.050116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6fecbff6-91ae-4bae-86c9-b520d231c4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:13.067574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 824/1636] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:12:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:13.081245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1b2c6-4edf-4b8b-ae1a-3b20fea806c0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:13.099567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1b2c6-4edf-4b8b-ae1a-3b20fea806c0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:13.109918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80a73f-6a83-47eb-b895-09687a593a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:13.110660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80a73f-6a83-47eb-b895-09687a593a9f service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:13.110997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80a73f-6a83-47eb-b895-09687a593a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:13.111276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80a73f-6a83-47eb-b895-09687a593a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:13.134222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80a73f-6a83-47eb-b895-09687a593a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:13.135678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80a73f-6a83-47eb-b895-09687a593a9f service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:13.173121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 813/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:14.946912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecbff6-91ae-4bae-86c9-b520d231c4e4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:15.130368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecbff6-91ae-4bae-86c9-b520d231c4e4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:15.143257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:15.143257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:15.143257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:15.143257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] PATH_INFO: `/v3/users/d35f22e67dbc4f9e9144596860b21e79` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:15.145362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35f22e67dbc4f9e9144596860b21e79)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:15.146502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:12:15.146502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 13 14:12:15.146502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: Sep 13 14:12:15.148083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:15.356764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35f22e67dbc4f9e9144596860b21e79'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:15.392557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35f22e67dbc4f9e9144596860b21e79'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:15.469484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35f22e67dbc4f9e9144596860b21e79'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:15.495201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35f22e67dbc4f9e9144596860b21e79'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:15.495932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35f22e67dbc4f9e9144596860b21e79'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:15.513311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 825/1638] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:14 2022] DELETE /identity/v3/users/d35f22e67dbc4f9e9144596860b21e79 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:15.515070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fc0c-fb9b-4767-a663-de5d5d165549 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:15.516059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fc0c-fb9b-4767-a663-de5d5d165549 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:15.516715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fc0c-fb9b-4767-a663-de5d5d165549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:15.571991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f625fc0c-fb9b-4767-a663-de5d5d165549 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:15.573869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f625fc0c-fb9b-4767-a663-de5d5d165549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:15.768629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 814/1639] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:15.805811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:15.807383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:15.953122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809ed27-2635-49af-8275-dec8a80ed966 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:15.962327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:15.964231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:15.964806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:15.965339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:15.985470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:15.988689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:16.007976 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 826/1640] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:16.454814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:16.455368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:16.455368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:16.508736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] MFA Rules not processed for user `92c30fba107b4dfbb7b7b1582cee2ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:16.510042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:16.651640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 815/1641] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:16.672234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:16.672234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:16.689367 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:16.832486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59cc44-3299-437e-8a1a-5b9e4f79c50e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:16.850516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db96ca3-a9b8-47a8-9ce3-067802dfdde4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:16.851222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db96ca3-a9b8-47a8-9ce3-067802dfdde4 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:16.851496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db96ca3-a9b8-47a8-9ce3-067802dfdde4 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:16.851761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db96ca3-a9b8-47a8-9ce3-067802dfdde4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:16.866640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db96ca3-a9b8-47a8-9ce3-067802dfdde4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:16.872729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db96ca3-a9b8-47a8-9ce3-067802dfdde4 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:16.911892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 827/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:17.321998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:17.324276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:17.363352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6c525-91f5-4632-a62e-d6592a3e9b60 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:17.390988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:17.392225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:17.392601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:17.392910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] PATH_INFO: `/v3/projects/5a575c06adfd4cb8a6722de49994004c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:17.394859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a575c06adfd4cb8a6722de49994004c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:17.397990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: WARNING py.warnings [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 14:12:17.397990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: warnings.warn(msg) Sep 13 14:12:17.397990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: Sep 13 14:12:17.404079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:17.538169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a575c06adfd4cb8a6722de49994004c', 'request_id': 'req-f1a77275-d815-42b8-baaf-8688294e7b33'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:17.566798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:17.571260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a575c06adfd4cb8a6722de49994004c', 'request_id': 'req-f1a77275-d815-42b8-baaf-8688294e7b33'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:17.587169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a575c06adfd4cb8a6722de49994004c', 'request_id': 'req-f1a77275-d815-42b8-baaf-8688294e7b33'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:17.632224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] The token cache is being invalidate because project 5a575c06adfd4cb8a6722de49994004c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:17.632732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:17.635755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f1a77275-d815-42b8-baaf-8688294e7b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:17.644212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 816/1643] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:17 2022] DELETE /identity/v3/projects/5a575c06adfd4cb8a6722de49994004c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:17.645968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8b5cc-f8aa-4f3f-9e56-c103ff6167f0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:17.647189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8b5cc-f8aa-4f3f-9e56-c103ff6167f0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:17.649650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8b5cc-f8aa-4f3f-9e56-c103ff6167f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:17.697428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f4e8b5cc-f8aa-4f3f-9e56-c103ff6167f0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:17.701585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f4e8b5cc-f8aa-4f3f-9e56-c103ff6167f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:17.951164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 828/1644] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:17.966130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c05b7-48dd-4721-86b1-083a4f71b2cb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:17.966926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c05b7-48dd-4721-86b1-083a4f71b2cb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:17.975219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c05b7-48dd-4721-86b1-083a4f71b2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.005354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8b9c05b7-48dd-4721-86b1-083a4f71b2cb None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:18.007593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8b9c05b7-48dd-4721-86b1-083a4f71b2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:18.021307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 817/1645] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:18.029064 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8b5cc-f8aa-4f3f-9e56-c103ff6167f0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:18.046997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8b5cc-f8aa-4f3f-9e56-c103ff6167f0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:18.082671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:18.083722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:18.084263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:18.084805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.085676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:18.086258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:18.089120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:18.110163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 829/1646] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:18.114142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c05b7-48dd-4721-86b1-083a4f71b2cb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:18.129999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c05b7-48dd-4721-86b1-083a4f71b2cb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:18.144802 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:18.145677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:18.146170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:18.147258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.148612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:18.156184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:18.234447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:18.235454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 818/1647] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:18 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:18.253918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e012c1-c0e8-4995-ac1f-9634aff69d7a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:18.435183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:18.435814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:18.436090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:18.436362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.437394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:18.439437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:18.489906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 830/1648] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:18 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:18.494287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:18.508175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c0dd-7e95-42b7-8e76-df518de60d31 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:18.516474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:18.517252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:18.517817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:18.518294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.521068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:18.524229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:18.539128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 819/1649] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:18.540589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:18.557180 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c924a9-9ad8-41d4-b179-389e647c3af0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:18.576241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5411cc-98c2-4c57-8004-8c5d870ece91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:18.577179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5411cc-98c2-4c57-8004-8c5d870ece91 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:18.577642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5411cc-98c2-4c57-8004-8c5d870ece91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:18.578116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5411cc-98c2-4c57-8004-8c5d870ece91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.579155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5411cc-98c2-4c57-8004-8c5d870ece91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:18.581190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5411cc-98c2-4c57-8004-8c5d870ece91 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:18.583836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:18.598865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 831/1650] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:18.599937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:18.632336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f209f2-ad3d-4c5a-96bf-90a7c7a7aa8f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:18.652307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:18.653562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:18.654118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:18.654635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] PATH_INFO: `/v3/projects/1e0603070f9a4e8b86a53fc426adb511/users/d5b73955d20e497c96716813be237e9f/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.682212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0603070f9a4e8b86a53fc426adb511, user_id=d5b73955d20e497c96716813be237e9f, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:18.686369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:18.722699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 820/1651] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:18 2022] PUT /identity/v3/projects/1e0603070f9a4e8b86a53fc426adb511/users/d5b73955d20e497c96716813be237e9f/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:18.736462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde69cf7-9637-4dcf-8b44-44b1749a3d0a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:18.737132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde69cf7-9637-4dcf-8b44-44b1749a3d0a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:18.737419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde69cf7-9637-4dcf-8b44-44b1749a3d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:18.864141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bde69cf7-9637-4dcf-8b44-44b1749a3d0a None None] MFA Rules not processed for user `d5b73955d20e497c96716813be237e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:18.865958 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bde69cf7-9637-4dcf-8b44-44b1749a3d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:19.089383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 832/1652] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:19.099466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:19.102238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:19.262204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ede9ec-c41d-45d2-986b-a79dea043030 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:19.269106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82705b-ace9-4185-9915-34b2f0290467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:19.270512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82705b-ace9-4185-9915-34b2f0290467 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:19.271157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82705b-ace9-4185-9915-34b2f0290467 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:19.272244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82705b-ace9-4185-9915-34b2f0290467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:19.282150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82705b-ace9-4185-9915-34b2f0290467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:19.283672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82705b-ace9-4185-9915-34b2f0290467 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:19.306592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 821/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:19.318360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde69cf7-9637-4dcf-8b44-44b1749a3d0a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:19.465681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde69cf7-9637-4dcf-8b44-44b1749a3d0a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:19.472721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:19.473384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:19.477259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:19.477259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] PATH_INFO: `/v3/users/d5b73955d20e497c96716813be237e9f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:19.479983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b73955d20e497c96716813be237e9f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:19.486521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:19.614630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b73955d20e497c96716813be237e9f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:19.653538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b73955d20e497c96716813be237e9f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:19.691247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b73955d20e497c96716813be237e9f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:19.713278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b73955d20e497c96716813be237e9f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:19.714049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b73955d20e497c96716813be237e9f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:19.723057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 833/1654] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:19 2022] DELETE /identity/v3/users/d5b73955d20e497c96716813be237e9f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:19.727970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:19.728268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:19.728989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:19.761083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:19.763050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:19.906482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 822/1655] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:20.110675 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:20.112899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:20.145431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:20.345888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb13a0-e24b-4239-bdf0-b1369dfaeb69 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:20.363859 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:20.364599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:20.366115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:20.366506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:20.387179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:20.389349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:20.426075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 834/1656] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:22.738199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:22.740455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:22.764638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b7f83-5ebf-4f09-9d4a-3b4f6097b82e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:22.788298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:22.789530 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:22.790013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:22.790441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] PATH_INFO: `/v3/users/c90218bf52dd428c979cba8eda045eb2` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:22.792649 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90218bf52dd428c979cba8eda045eb2)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:22.795255 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:22.830734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:22.883583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:22.887230 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:22.914000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddabac9-4417-45af-b58d-e9d0fb5c2256 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:22.921391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90218bf52dd428c979cba8eda045eb2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:22.930049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:22.932519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:22.933630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:22.935547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] PATH_INFO: `/v3/projects/1e0603070f9a4e8b86a53fc426adb511` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:22.938412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0603070f9a4e8b86a53fc426adb511)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:22.948673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:22.950324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90218bf52dd428c979cba8eda045eb2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.018290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90218bf52dd428c979cba8eda045eb2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.023586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90218bf52dd428c979cba8eda045eb2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.025053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-89699afa-844b-40de-84c2-e0b9888bb5f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90218bf52dd428c979cba8eda045eb2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.067612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 823/1657] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:22 2022] DELETE /identity/v3/users/c90218bf52dd428c979cba8eda045eb2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:23.069771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.070229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.070761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.085128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0603070f9a4e8b86a53fc426adb511', 'request_id': 'req-b56b4fd4-4131-46a8-8da7-22c0e947e795'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.097317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:23.098950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:23.099443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0603070f9a4e8b86a53fc426adb511', 'request_id': 'req-b56b4fd4-4131-46a8-8da7-22c0e947e795'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.111553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0603070f9a4e8b86a53fc426adb511', 'request_id': 'req-b56b4fd4-4131-46a8-8da7-22c0e947e795'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.136128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] The token cache is being invalidate because project 1e0603070f9a4e8b86a53fc426adb511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:23.136700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.138179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b56b4fd4-4131-46a8-8da7-22c0e947e795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:23.146501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 835/1658] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:22 2022] DELETE /identity/v3/projects/1e0603070f9a4e8b86a53fc426adb511 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:23.149384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe3a11-8360-4415-baa1-3fcc1858684c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.149839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe3a11-8360-4415-baa1-3fcc1858684c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.150297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe3a11-8360-4415-baa1-3fcc1858684c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.190966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2fbe3a11-8360-4415-baa1-3fcc1858684c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:23.192343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2fbe3a11-8360-4415-baa1-3fcc1858684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:23.329023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 824/1659] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:23.348239 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:23.350589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:23.377482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 836/1660] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:23.379969 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b254e-4185-473f-ba2c-c25d8f5a1719 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.380345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b254e-4185-473f-ba2c-c25d8f5a1719 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.381485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b254e-4185-473f-ba2c-c25d8f5a1719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.438994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-006b254e-4185-473f-ba2c-c25d8f5a1719 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:23.439735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-006b254e-4185-473f-ba2c-c25d8f5a1719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:23.467833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 837/1661] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:23.469017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b254e-4185-473f-ba2c-c25d8f5a1719 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:23.486869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8be51a-0519-4cf8-966b-10129f0413fe None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:23.491961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b254e-4185-473f-ba2c-c25d8f5a1719 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:23.498075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:23.500012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.500448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:23.500633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.501762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.501762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.502056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.502749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:23.503096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:23.503559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.507379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:23.538305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:23.541964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 838/1662] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:12:23.543461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:23.543895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:23.564360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 825/1663] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:12:23.568880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5326ea-80a1-49ff-a30b-2eb7452f2574 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:23.583554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e2a7-e852-48d9-8e60-d9acb1f9db9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:23.583554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e2a7-e852-48d9-8e60-d9acb1f9db9b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.584395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e2a7-e852-48d9-8e60-d9acb1f9db9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.584809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e2a7-e852-48d9-8e60-d9acb1f9db9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.585487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e2a7-e852-48d9-8e60-d9acb1f9db9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:23.589103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e2a7-e852-48d9-8e60-d9acb1f9db9b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:23.622229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:23.670342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 839/1664] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:23 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:23.672052 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:23.697554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa4f19-36a1-4cf4-b8f0-4edcea1a1bb0 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:23.718661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.719268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.719689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.820511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] MFA Rules not processed for user `17e8dbc75366467f8fc6f0681e722751`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:23.825081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:23.849311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:23.898480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:23.899366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:23.899807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:23.900311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:23.901283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:23.904067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:24.000318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 826/1665] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:23 2022] POST /identity/v3/users => generated 289 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:24.036005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:24.087010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 840/1666] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:24.092058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8d42d-7cb9-4eef-b5ae-05b4f763e25c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:24.095551 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:24.097924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:24.129995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:24.131198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:24.131480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:24.131884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:24.132546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:24.144789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:24.197422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 827/1667] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:24.202107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:24.251907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed66b8-e08a-4ded-acac-f144ff6751cf admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:24.281393 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:24.286825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:24.287252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:24.288007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:24.288719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:24.290506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:24.362631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 828/1668] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:24.366605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:24.386902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37321261-9fb0-432f-a0f3-2570fdf6b4fd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:24.403412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ddfc-c04a-491d-a671-378dd480b52b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:24.405704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:24.408856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:24.409143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:24.409421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] PATH_INFO: `/v3/projects/b75869fe5009408383d57e72c4c5b953/users/4866f815cf7044d498685a12c5d31df0/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:24.433438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4931bb1-a9bd-46f3-a598-7acaad7e49eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:24.434221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4931bb1-a9bd-46f3-a598-7acaad7e49eb service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:24.438748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4931bb1-a9bd-46f3-a598-7acaad7e49eb service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:24.439050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4931bb1-a9bd-46f3-a598-7acaad7e49eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:24.468650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4931bb1-a9bd-46f3-a598-7acaad7e49eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:24.472054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4931bb1-a9bd-46f3-a598-7acaad7e49eb service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:24.473026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75869fe5009408383d57e72c4c5b953, user_id=4866f815cf7044d498685a12c5d31df0, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:24.476601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:24.492416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 841/1669] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:24.503469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 829/1670] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:24 2022] PUT /identity/v3/projects/b75869fe5009408383d57e72c4c5b953/users/4866f815cf7044d498685a12c5d31df0/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:24.513838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb2527-0df9-4b9a-a60c-73bea37f5ce8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:24.514287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb2527-0df9-4b9a-a60c-73bea37f5ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:24.514626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb2527-0df9-4b9a-a60c-73bea37f5ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:24.585274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4adb2527-0df9-4b9a-a60c-73bea37f5ce8 None None] MFA Rules not processed for user `4866f815cf7044d498685a12c5d31df0`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:24.587584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4adb2527-0df9-4b9a-a60c-73bea37f5ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:24.719915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:24.738285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 842/1671] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:24.750359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:24.753257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:24.908242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006204b-4acd-46c9-9a60-43258150c8e8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:24.917861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d184fe-33a0-4572-a551-811739d95d86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7214e6addc646a899efb82283ededc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:24.918187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d184fe-33a0-4572-a551-811739d95d86 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:24.918563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d184fe-33a0-4572-a551-811739d95d86 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:24.918849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d184fe-33a0-4572-a551-811739d95d86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:24.954913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d184fe-33a0-4572-a551-811739d95d86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:24.966644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d184fe-33a0-4572-a551-811739d95d86 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:24.990499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 830/1672] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:25.703535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:25.704031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:25.704125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:25.819139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] MFA Rules not processed for user `961012089bd147ecab4fcdf5ea3ae4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:25.820638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:25.935499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 843/1673] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:25.946442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:25.946975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:25.946975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:25.984363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] MFA Rules not processed for user `961012089bd147ecab4fcdf5ea3ae4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:25.985604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:26.004515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 831/1674] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:26.016935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:26.018397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:26.191692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862f4c3-faa3-4293-aafe-5d4ea14fbbb6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:26.215628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9973e-52fd-42b0-a0a4-d237cb82cd2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:26.218516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9973e-52fd-42b0-a0a4-d237cb82cd2e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:26.218516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9973e-52fd-42b0-a0a4-d237cb82cd2e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:26.221544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9973e-52fd-42b0-a0a4-d237cb82cd2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:26.256258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9973e-52fd-42b0-a0a4-d237cb82cd2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:26.265296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9973e-52fd-42b0-a0a4-d237cb82cd2e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:26.295593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 844/1675] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:26.518462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:26.519957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:26.680467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c4bde-ef54-42a3-a9a1-a368a467311f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:26.688348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:26.689955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:26.690603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:26.691280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] PATH_INFO: `/v3/projects/88abd18fe3a84f619312d663b59da63f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:26.695139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88abd18fe3a84f619312d663b59da63f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:26.698469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:26.701358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:26.813602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88abd18fe3a84f619312d663b59da63f', 'request_id': 'req-109cd3c2-e30a-460d-96a9-adf7321b3c26'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:26.832355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88abd18fe3a84f619312d663b59da63f', 'request_id': 'req-109cd3c2-e30a-460d-96a9-adf7321b3c26'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:26.851161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88abd18fe3a84f619312d663b59da63f', 'request_id': 'req-109cd3c2-e30a-460d-96a9-adf7321b3c26'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:26.876643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] The token cache is being invalidate because project 88abd18fe3a84f619312d663b59da63f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:26.877226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:26.879769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-109cd3c2-e30a-460d-96a9-adf7321b3c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:26.886912 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 832/1676] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:26 2022] DELETE /identity/v3/projects/88abd18fe3a84f619312d663b59da63f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:26.888932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9516451-c83d-4647-b814-42eb163fc621 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:26.889440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9516451-c83d-4647-b814-42eb163fc621 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:26.889692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9516451-c83d-4647-b814-42eb163fc621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:26.935964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d9516451-c83d-4647-b814-42eb163fc621 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:26.937489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9516451-c83d-4647-b814-42eb163fc621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:27.003439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852588d-d65d-42cb-a004-274f875555d3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.003882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852588d-d65d-42cb-a004-274f875555d3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.004724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852588d-d65d-42cb-a004-274f875555d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.113017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 845/1677] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:27.114937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beecee5-c1bd-49f5-9bff-950e6e9b7cbe None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.115570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beecee5-c1bd-49f5-9bff-950e6e9b7cbe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.116263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beecee5-c1bd-49f5-9bff-950e6e9b7cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.147858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2beecee5-c1bd-49f5-9bff-950e6e9b7cbe None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:27.149425 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2beecee5-c1bd-49f5-9bff-950e6e9b7cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:27.162772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 846/1678] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:27.168260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beecee5-c1bd-49f5-9bff-950e6e9b7cbe None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:27.183743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:27.188359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beecee5-c1bd-49f5-9bff-950e6e9b7cbe None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:27.192170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0852588d-d65d-42cb-a004-274f875555d3 None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:27.193978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0852588d-d65d-42cb-a004-274f875555d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:27.197413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:27.198308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.199077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.199554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.200388 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:27.200922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:27.203028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:27.230493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 847/1679] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:27.231471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:27.248317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fcd4e-7376-48c0-b834-1c0433b80587 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:27.257518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:27.259746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.260040 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.260322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.262833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:27.267467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:27.326788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 848/1680] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:27 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:27.333183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:27.420182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 833/1681] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:12:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:27.542851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d60f86-28a2-4b46-ac76-9e2c31066ee4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:27.557963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754bab0-a7a5-49a6-94ab-158a551f0ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:27.559294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754bab0-a7a5-49a6-94ab-158a551f0ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.559689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754bab0-a7a5-49a6-94ab-158a551f0ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.560070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754bab0-a7a5-49a6-94ab-158a551f0ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.562050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754bab0-a7a5-49a6-94ab-158a551f0ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:27.564133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754bab0-a7a5-49a6-94ab-158a551f0ec7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:27.601988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 849/1682] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:27 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:27.602370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852588d-d65d-42cb-a004-274f875555d3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:27.616172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852588d-d65d-42cb-a004-274f875555d3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:27.625370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:27.626186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.626583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.626935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.627845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:27.629471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:27.654765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665899-5fe5-4ac1-a4c0-e5c693cf99e5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.655944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665899-5fe5-4ac1-a4c0-e5c693cf99e5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.656231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665899-5fe5-4ac1-a4c0-e5c693cf99e5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.657837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 850/1683] 10.222.0.20 () {54 vars in 912 bytes} [Tue Sep 13 14:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:27.674681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458608b-d1c7-47cf-b8b8-670af529bba7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.674968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458608b-d1c7-47cf-b8b8-670af529bba7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.675330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458608b-d1c7-47cf-b8b8-670af529bba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.682713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 834/1684] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:27.697410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:27.729345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f1f53-a5d0-4dab-9291-9c36b94caf7f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:27.742754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:27.747934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.748128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.748468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:27.749616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:27.752762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:27.777068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 835/1685] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:27.780621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:27.782229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f458608b-d1c7-47cf-b8b8-670af529bba7 None None] MFA Rules not processed for user `11a418a2a35146b59c433421fe507500`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:27.783120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:27.785974 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f458608b-d1c7-47cf-b8b8-670af529bba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:27.912297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:27.939930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 851/1686] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:12:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:27.941328 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458608b-d1c7-47cf-b8b8-670af529bba7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:27.970567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458608b-d1c7-47cf-b8b8-670af529bba7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:27.981033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90fec1-1903-4bcc-b76a-394021cf1300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:27.982147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90fec1-1903-4bcc-b76a-394021cf1300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:27.982628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90fec1-1903-4bcc-b76a-394021cf1300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:27.983037 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90fec1-1903-4bcc-b76a-394021cf1300 admin admin] PATH_INFO: `/v3/projects/2274dd669511440dbb08417a51b043d2/users/d521c514342b4fc18fb1d1d970a7172f/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:28.023663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90fec1-1903-4bcc-b76a-394021cf1300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2274dd669511440dbb08417a51b043d2, user_id=d521c514342b4fc18fb1d1d970a7172f, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:28.026948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90fec1-1903-4bcc-b76a-394021cf1300 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:28.029105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ae246-d658-4e7a-ab47-6bb22223c635 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:28.043187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48631b9-a9bd-42c4-8769-c210b7d844ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:28.044253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48631b9-a9bd-42c4-8769-c210b7d844ed service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:28.044540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48631b9-a9bd-42c4-8769-c210b7d844ed service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:28.045616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48631b9-a9bd-42c4-8769-c210b7d844ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:28.061990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 852/1687] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:27 2022] PUT /identity/v3/projects/2274dd669511440dbb08417a51b043d2/users/d521c514342b4fc18fb1d1d970a7172f/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:28.074538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623021e7-10a7-406c-9bae-2a600afd5813 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:28.074538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623021e7-10a7-406c-9bae-2a600afd5813 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:28.075063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623021e7-10a7-406c-9bae-2a600afd5813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:28.144489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-623021e7-10a7-406c-9bae-2a600afd5813 None None] MFA Rules not processed for user `d521c514342b4fc18fb1d1d970a7172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:28.147366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-623021e7-10a7-406c-9bae-2a600afd5813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:28.189819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:28.193496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48631b9-a9bd-42c4-8769-c210b7d844ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:28.201308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48631b9-a9bd-42c4-8769-c210b7d844ed service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:28.248199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 836/1688] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:12:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:12:28.254485 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:28.254695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:28.254976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:28.291968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 853/1689] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:28.296571 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623021e7-10a7-406c-9bae-2a600afd5813 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:28.332733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] MFA Rules not processed for user `92c30fba107b4dfbb7b7b1582cee2ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:28.334848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:28.462272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623021e7-10a7-406c-9bae-2a600afd5813 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:28.478535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:28.479322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:28.479652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:28.479973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:28.480799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:28.483016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:28.540356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 854/1690] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:28 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:28.542566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:28.578115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 837/1691] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:28.579855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:28.590913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:28.801291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e6b7f-52e7-4cf7-8086-a760adfc8fb1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:28.845862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc457486-e128-4ef1-85ee-9ad5afa1dd28 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:28.864258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:28.866704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:28.867233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:28.867878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:28.868892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:28.869391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:28.873090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:28.921146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 855/1692] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:28 2022] POST /identity/v3/users => generated 289 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:28.925233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:28.946648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8ace-b9c9-441c-ab5f-37cafa9f47c8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:28.971714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:28.972512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:28.976289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:28.976721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:28.977495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:28.979277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:29.020164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 856/1693] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:12:29.022429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:29.038605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5d8b3-ed9d-4e6e-8e5e-5996eed04526 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:29.048451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:29.051466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:29.053567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:29.054093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] PATH_INFO: `/v3/projects/5cd2ea434a8742b395af5f2dcda0cd5e/users/9f593cd08c7544b9a2724f3ed085c187/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:29.058674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:29.060259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:29.060693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:29.061109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:29.078691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:29.080171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:29.090727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd2ea434a8742b395af5f2dcda0cd5e, user_id=9f593cd08c7544b9a2724f3ed085c187, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:29.096481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:29.119228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:29.122029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 838/1694] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:29.129277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 857/1695] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:29 2022] PUT /identity/v3/projects/5cd2ea434a8742b395af5f2dcda0cd5e/users/9f593cd08c7544b9a2724f3ed085c187/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:29.133455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:29.178062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a69e44-7dd6-410d-b728-d1f73d8a7a09 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:29.379465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a0f8-6b2f-48e7-a5b5-959796596d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:29.380845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a0f8-6b2f-48e7-a5b5-959796596d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:29.386139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a0f8-6b2f-48e7-a5b5-959796596d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:29.386658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a0f8-6b2f-48e7-a5b5-959796596d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:29.388738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2a0f8-6b2f-48e7-a5b5-959796596d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:29.392778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2a0f8-6b2f-48e7-a5b5-959796596d4e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:29.419995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:29.420635 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 839/1696] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:29.444695 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fba7bc-7d93-4929-a1a6-413c05b50d3b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:29.463375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:29.464080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:29.464365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:29.464647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] PATH_INFO: `/v3/projects/5cd2ea434a8742b395af5f2dcda0cd5e/users/9f593cd08c7544b9a2724f3ed085c187/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:29.468819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd2ea434a8742b395af5f2dcda0cd5e, user_id=9f593cd08c7544b9a2724f3ed085c187, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:29.486681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:29.522315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 858/1697] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:29 2022] PUT /identity/v3/projects/5cd2ea434a8742b395af5f2dcda0cd5e/users/9f593cd08c7544b9a2724f3ed085c187/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:29.523754 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:29.524088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:29.524302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:29.585593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] MFA Rules not processed for user `9f593cd08c7544b9a2724f3ed085c187`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:29.587721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:29.796170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 840/1698] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:29.805693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:29.811235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:29.937138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373a8a0-57d3-4e39-9d57-3438d41f6d30 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:29.941164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d7a2c-ea74-4979-b4af-11cc1f2ba763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:29.941810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d7a2c-ea74-4979-b4af-11cc1f2ba763 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:29.942094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d7a2c-ea74-4979-b4af-11cc1f2ba763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:29.942370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d7a2c-ea74-4979-b4af-11cc1f2ba763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:29.954159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d7a2c-ea74-4979-b4af-11cc1f2ba763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:29.956279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d7a2c-ea74-4979-b4af-11cc1f2ba763 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:29.978298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 859/1699] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:30.701284 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:30.703599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:30.871735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f074bd-9375-4159-be0c-d09691cf451d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:30.872012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f074bd-9375-4159-be0c-d09691cf451d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:30.872299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f074bd-9375-4159-be0c-d09691cf451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:30.913268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-23f074bd-9375-4159-be0c-d09691cf451d None None] MFA Rules not processed for user `961012089bd147ecab4fcdf5ea3ae4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:30.921987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-23f074bd-9375-4159-be0c-d09691cf451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:30.933637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:30.986006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a54f4-1c31-4a99-a435-57b1ac824e00 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:30.998863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:31.001317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:31.002829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:31.004253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:31.012079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:31.056365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:31.058289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:31.110410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 841/1700] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:31.138704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 860/1701] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:31.150015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:31.211819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ebf8e-f9be-4661-8570-14940f5d9502 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:31.212136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ebf8e-f9be-4661-8570-14940f5d9502 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:31.212384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ebf8e-f9be-4661-8570-14940f5d9502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:31.245809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f79ebf8e-f9be-4661-8570-14940f5d9502 None None] MFA Rules not processed for user `17e8dbc75366467f8fc6f0681e722751`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:31.248497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f79ebf8e-f9be-4661-8570-14940f5d9502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:31.319773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6703023-6656-474c-a6a2-6b1877b57f87 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:31.334411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:31.335136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:31.335416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:31.335690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] PATH_INFO: `/v3/users/d521c514342b4fc18fb1d1d970a7172f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:31.339061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d521c514342b4fc18fb1d1d970a7172f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:31.341010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:31.378171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 861/1702] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:31.462557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd521c514342b4fc18fb1d1d970a7172f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:31.476618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd521c514342b4fc18fb1d1d970a7172f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:31.554573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd521c514342b4fc18fb1d1d970a7172f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:31.556101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd521c514342b4fc18fb1d1d970a7172f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:31.557360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd521c514342b4fc18fb1d1d970a7172f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:31.612356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 842/1703] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:31 2022] DELETE /identity/v3/users/d521c514342b4fc18fb1d1d970a7172f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:31.616787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bb611-1208-4bd8-a759-40bd3460d58d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:31.617075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bb611-1208-4bd8-a759-40bd3460d58d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:31.617353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bb611-1208-4bd8-a759-40bd3460d58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:31.643209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-102bb611-1208-4bd8-a759-40bd3460d58d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:31.651905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-102bb611-1208-4bd8-a759-40bd3460d58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:31.764474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:31.839057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 862/1704] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:31.884467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:31.886246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:32.004831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424081e5-d01b-4fd8-8f1c-d424455169b9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:32.005166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424081e5-d01b-4fd8-8f1c-d424455169b9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:32.005452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424081e5-d01b-4fd8-8f1c-d424455169b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:32.099286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-424081e5-d01b-4fd8-8f1c-d424455169b9 None None] MFA Rules not processed for user `a22a06667e10411e8d8ce92db4536a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:32.119165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-424081e5-d01b-4fd8-8f1c-d424455169b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:32.119379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01cd474-822b-4870-a141-d0cf2261a931 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:32.129126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5558223-01df-4d86-9e1b-48d0dd9e82c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:32.131881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5558223-01df-4d86-9e1b-48d0dd9e82c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:32.132404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5558223-01df-4d86-9e1b-48d0dd9e82c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:32.134927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5558223-01df-4d86-9e1b-48d0dd9e82c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:32.167834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5558223-01df-4d86-9e1b-48d0dd9e82c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:32.174984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5558223-01df-4d86-9e1b-48d0dd9e82c3 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:32.188499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:32.205494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 843/1705] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:12:32.207119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ff4dc-bae8-4236-acfb-883a3d2a0311 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:32.207385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ff4dc-bae8-4236-acfb-883a3d2a0311 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:32.211624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ff4dc-bae8-4236-acfb-883a3d2a0311 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:32.213338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 844/1706] 10.222.0.20 () {54 vars in 912 bytes} [Tue Sep 13 14:12:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:32.220332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:32.220638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:32.220897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:32.280500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] MFA Rules not processed for user `11a418a2a35146b59c433421fe507500`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:32.281838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:32.313218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 845/1707] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:12:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:32.419302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:32.422557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:32.439254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 863/1708] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:32.454133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28234c2-1b66-4dba-a1e9-a4497408b66e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:32.459687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:32.460608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:32.460957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:32.461308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:32.475349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:32.476971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:32.489175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 846/1709] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:33.394839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:33.395356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:33.395891 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:33.493296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] MFA Rules not processed for user `e35dfe8442964bd793276386c2fe6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:33.495441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:33.603394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:33.604889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:33.648505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acea46-9e64-4bf1-9242-c8e8d172fb66 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:33.669403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:33.670356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:33.670584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:33.670865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] PATH_INFO: `/v3/users/a2713c44481840d290cfda681adb9d01` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:33.674513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2713c44481840d290cfda681adb9d01)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:33.676493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:33.852445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 864/1710] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:33.853637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:33.855749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:33.865570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2713c44481840d290cfda681adb9d01'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:33.899380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2713c44481840d290cfda681adb9d01'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:33.906133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6100fc2-40a2-4f49-af0d-1c8b01aab1d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:33.918378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:33.919165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:33.919525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:33.919809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] PATH_INFO: `/v3/projects/2274dd669511440dbb08417a51b043d2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:33.922189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2274dd669511440dbb08417a51b043d2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:33.930035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:33.994646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2713c44481840d290cfda681adb9d01'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:33.994646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2713c44481840d290cfda681adb9d01'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:33.995591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-345e2e6f-14ed-4cf2-bcb5-8956d6fea935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2713c44481840d290cfda681adb9d01'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.067299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 847/1711] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:33 2022] DELETE /identity/v3/users/a2713c44481840d290cfda681adb9d01 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:34.067299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22d4a-dbf0-481c-aba5-9192f8183e71 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:34.070287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22d4a-dbf0-481c-aba5-9192f8183e71 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:34.070671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22d4a-dbf0-481c-aba5-9192f8183e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:34.120403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2274dd669511440dbb08417a51b043d2', 'request_id': 'req-2f255281-7295-4442-a3e4-0c2f943b67e6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.120588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-80a22d4a-dbf0-481c-aba5-9192f8183e71 None None] MFA Rules not processed for user `e35dfe8442964bd793276386c2fe6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:34.126541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-80a22d4a-dbf0-481c-aba5-9192f8183e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:34.134977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:34.154333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2274dd669511440dbb08417a51b043d2', 'request_id': 'req-2f255281-7295-4442-a3e4-0c2f943b67e6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.166428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2274dd669511440dbb08417a51b043d2', 'request_id': 'req-2f255281-7295-4442-a3e4-0c2f943b67e6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.207708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] The token cache is being invalidate because project 2274dd669511440dbb08417a51b043d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:34.208103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.211311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.216573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 865/1712] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:33 2022] DELETE /identity/v3/projects/2274dd669511440dbb08417a51b043d2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:34.217479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:34.227314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:34.323732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 848/1713] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:34.325504 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:34.325980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:34.326364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:34.357037 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:34.361890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:34.548805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f255281-7295-4442-a3e4-0c2f943b67e6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:34.559492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:34.560523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:34.561440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:34.562237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:34.565359 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:34.600569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:34.629668 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 849/1714] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:34.638030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:34.640005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:34.654907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448089a3-1b33-4b3a-a123-5b3a4c19a832 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:34.660941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:34.662290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:34.662577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:34.662843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] PATH_INFO: `/v3/users/9f593cd08c7544b9a2724f3ed085c187` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:34.664366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f593cd08c7544b9a2724f3ed085c187)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:34.667444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:34.760405 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f593cd08c7544b9a2724f3ed085c187'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.767008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:34.768699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:34.802833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f593cd08c7544b9a2724f3ed085c187'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.821222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 866/1715] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:34.822298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:34.845527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:34.855984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f593cd08c7544b9a2724f3ed085c187'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.857040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f593cd08c7544b9a2724f3ed085c187'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.857880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f593cd08c7544b9a2724f3ed085c187'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:34.900908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 850/1716] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:34 2022] DELETE /identity/v3/users/9f593cd08c7544b9a2724f3ed085c187 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:35.099137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55893e65-00cb-4a0f-96e9-e43bce6cb8b7 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:35.236014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:35.274442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb8389-986b-4735-9f75-7709443fbfc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:35.275062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb8389-986b-4735-9f75-7709443fbfc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:35.275343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb8389-986b-4735-9f75-7709443fbfc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:35.275684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb8389-986b-4735-9f75-7709443fbfc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:35.287681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e7cb8389-986b-4735-9f75-7709443fbfc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:35.491574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb8389-986b-4735-9f75-7709443fbfc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:35.493139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb8389-986b-4735-9f75-7709443fbfc2 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:35.519992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 867/1717] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:35.550015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:35.551807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:35.738306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:35.738792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:35.738989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:35.783307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:35.784563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:35.859878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bcc5c-15a3-4a0d-b706-519b3a7dde49 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:35.880113 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:35.880804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:35.881088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:35.881387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:35.894505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:35.917085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 868/1718] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:12:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:35.940758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:35.942312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:36.095349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:36.096944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:36.133280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 851/1719] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:36.168010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb3dc0-bfa2-49f0-8a54-d5f76799dc4e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:36.182214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949f3ce-6e4b-4e8e-a589-5471493d9dc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:36.183289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949f3ce-6e4b-4e8e-a589-5471493d9dc5 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:36.183723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949f3ce-6e4b-4e8e-a589-5471493d9dc5 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:36.184139 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949f3ce-6e4b-4e8e-a589-5471493d9dc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:36.205161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949f3ce-6e4b-4e8e-a589-5471493d9dc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:36.207003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949f3ce-6e4b-4e8e-a589-5471493d9dc5 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:36.234635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 869/1720] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:12:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 14:12:36.748948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:36.751665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:36.767817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:36.788163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ce6da-6dbc-4e10-8bf8-ee0423b392b4 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:36.808324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:36.809440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:36.810025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:36.810460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] PATH_INFO: `/v3/projects/5cd2ea434a8742b395af5f2dcda0cd5e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:36.820015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd2ea434a8742b395af5f2dcda0cd5e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:36.827578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:36.977821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd2ea434a8742b395af5f2dcda0cd5e', 'request_id': 'req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:37.002693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd2ea434a8742b395af5f2dcda0cd5e', 'request_id': 'req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:37.040278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd2ea434a8742b395af5f2dcda0cd5e', 'request_id': 'req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:37.065956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] The token cache is being invalidate because project 5cd2ea434a8742b395af5f2dcda0cd5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:37.066563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:37.066923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8c1ad4af-50ba-4a12-a4da-8a20e3a9dcf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:37.068701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 852/1721] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:36 2022] DELETE /identity/v3/projects/5cd2ea434a8742b395af5f2dcda0cd5e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:37.074597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd01292-4dd6-4943-b4fb-94691029ef5c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:37.075099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd01292-4dd6-4943-b4fb-94691029ef5c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:37.075188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd01292-4dd6-4943-b4fb-94691029ef5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:37.142438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fbd01292-4dd6-4943-b4fb-94691029ef5c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:37.143740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fbd01292-4dd6-4943-b4fb-94691029ef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:37.230735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:37.468109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 870/1722] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:37.469988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560f1a8-a9c4-4c4e-86ec-9c66fea3c7dd None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:37.470469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560f1a8-a9c4-4c4e-86ec-9c66fea3c7dd None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:37.470876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560f1a8-a9c4-4c4e-86ec-9c66fea3c7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:37.567227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2560f1a8-a9c4-4c4e-86ec-9c66fea3c7dd None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:37.573458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2560f1a8-a9c4-4c4e-86ec-9c66fea3c7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:37.619115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 853/1723] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:37.623357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd01292-4dd6-4943-b4fb-94691029ef5c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:37.640023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd01292-4dd6-4943-b4fb-94691029ef5c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:37.655163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:37.655802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:37.656094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:37.656372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:37.657042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:37.657350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:37.659038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:37.705930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 871/1724] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:37.707702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560f1a8-a9c4-4c4e-86ec-9c66fea3c7dd None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:37.782089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560f1a8-a9c4-4c4e-86ec-9c66fea3c7dd None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:37.812630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:37.813338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:37.820132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:37.822672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:37.823384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:37.843972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:37.984375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 854/1725] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:37 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:12:37.987351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:38.011224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161d41b-65b8-43b5-b1f9-83c110820a7e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:38.284976 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:38.285842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:38.286873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:38.287465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:38.288364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:38.292841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:38.351862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 872/1726] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:37 2022] POST /identity/v3/users => generated 293 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:38.356456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:38.372329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf36a8-54c2-405e-9919-66850e98812b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:38.394506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:38.396437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:38.397090 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:38.397380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:38.399258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:38.403733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:38.437895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 855/1727] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:38.440729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:38.466187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27808e0a-e5e7-4d8b-9b08-6ffac8c64ab0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:38.498842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfb521-3ce6-4b4a-bc94-64afc0d12803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:38.499455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb521-3ce6-4b4a-bc94-64afc0d12803 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:38.499744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb521-3ce6-4b4a-bc94-64afc0d12803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:38.500200 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb521-3ce6-4b4a-bc94-64afc0d12803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:38.519047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfb521-3ce6-4b4a-bc94-64afc0d12803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:38.528403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfb521-3ce6-4b4a-bc94-64afc0d12803 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:38.565155 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 873/1728] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:38.569647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:38.595575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bd503-04d0-42bc-bf14-4965de7c7237 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:38.638169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcbe0fd-6716-4891-ac90-48a81b3c3a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:38.643409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbe0fd-6716-4891-ac90-48a81b3c3a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:38.647387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbe0fd-6716-4891-ac90-48a81b3c3a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:38.648130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcbe0fd-6716-4891-ac90-48a81b3c3a47 admin admin] PATH_INFO: `/v3/projects/2f376a605bc2430087314a29959cc86b/users/164bd6fe13664a66a23935c610cbbfd0/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:38.732187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbe0fd-6716-4891-ac90-48a81b3c3a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f376a605bc2430087314a29959cc86b, user_id=164bd6fe13664a66a23935c610cbbfd0, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:38.736639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcbe0fd-6716-4891-ac90-48a81b3c3a47 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:38.788702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 856/1729] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:38 2022] PUT /identity/v3/projects/2f376a605bc2430087314a29959cc86b/users/164bd6fe13664a66a23935c610cbbfd0/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:38.800752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:38.801394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:38.802338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:38.901660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] MFA Rules not processed for user `164bd6fe13664a66a23935c610cbbfd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:38.902903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:39.046956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352357e-d262-48b6-b70c-d8fdf2e6e74b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:39.047462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352357e-d262-48b6-b70c-d8fdf2e6e74b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:39.047756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352357e-d262-48b6-b70c-d8fdf2e6e74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:39.115235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3352357e-d262-48b6-b70c-d8fdf2e6e74b None None] MFA Rules not processed for user `11a418a2a35146b59c433421fe507500`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:39.116969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3352357e-d262-48b6-b70c-d8fdf2e6e74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:39.121411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 874/1730] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:39.124572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:39.127130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:39.172424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:39.371999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 857/1731] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 13 14:12:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:39.372998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352357e-d262-48b6-b70c-d8fdf2e6e74b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:39.470979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296626-0dfe-43b4-a2eb-a801503931c3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:39.498713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:39.500048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:39.500556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:39.501501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] PATH_INFO: `/v3/projects/ac73625cb9244518a2165a776a9819d5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:39.506656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac73625cb9244518a2165a776a9819d5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:39.515418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:39.588696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:39.684396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352357e-d262-48b6-b70c-d8fdf2e6e74b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:39.686501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac73625cb9244518a2165a776a9819d5', 'request_id': 'req-876e3222-2174-4da8-88d9-d1fe0bf43e22'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:39.702675 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac73625cb9244518a2165a776a9819d5', 'request_id': 'req-876e3222-2174-4da8-88d9-d1fe0bf43e22'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:39.714147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac73625cb9244518a2165a776a9819d5', 'request_id': 'req-876e3222-2174-4da8-88d9-d1fe0bf43e22'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:39.742522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] The token cache is being invalidate because project ac73625cb9244518a2165a776a9819d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:39.742826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:39.743780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:39.754306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 875/1732] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:39 2022] DELETE /identity/v3/projects/ac73625cb9244518a2165a776a9819d5 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:39.755955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:39.757588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:39.879196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:39.880646 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:39.881369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:39.888359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:39.890458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:39.893832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:39.978832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e3222-2174-4da8-88d9-d1fe0bf43e22 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:39.989642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 858/1733] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:39 2022] POST /identity/v3/projects => generated 385 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:39.990040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:39.993279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:39.993953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:39.998399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:39.998399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:40.000271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:40.003781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:40.058774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:40.059992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:40.126775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 876/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:40.127829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:40.145067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:40.216060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540b660-f1b8-447b-8889-b3f6f8fe9832 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:40.316587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99e169-18db-4df2-9460-8def16850acc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:40.332586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:40.332882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:40.334136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:40.335538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] PATH_INFO: `/v3/users/b05839c3a78b4593849b360f3573bee1` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:40.338490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b05839c3a78b4593849b360f3573bee1)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:40.341398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:40.363558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:40.397026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:40.397984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:40.400078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:40.400531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:40.401467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:40.407500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:40.481975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 877/1735] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:40 2022] POST /identity/v3/users => generated 299 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:40.484468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b05839c3a78b4593849b360f3573bee1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:40.487775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:40.507550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b05839c3a78b4593849b360f3573bee1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:40.517962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1042da-38f7-4463-840b-d317f1612bc4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:40.534383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:40.539306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:40.539840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:40.540383 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:40.541255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:40.543746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:40.549463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:40.554795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b05839c3a78b4593849b360f3573bee1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:40.557153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b05839c3a78b4593849b360f3573bee1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:40.562447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b05839c3a78b4593849b360f3573bee1'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:40.576486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 878/1736] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:40.579523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:40.588552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:40.598288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 859/1737] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:39 2022] DELETE /identity/v3/users/b05839c3a78b4593849b360f3573bee1 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:40.638684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e50f5-d084-474c-824c-3d34607f330e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:40.748032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:40.749157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:40.749642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:40.750085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] PATH_INFO: `/v3/projects/34a63b45a42b43aab2c14072a2c2cc4a/users/8024f8899f5c48d6a59a66d9fb96c868/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:40.773457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a63b45a42b43aab2c14072a2c2cc4a, user_id=8024f8899f5c48d6a59a66d9fb96c868, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:40.776661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:40.805782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 879/1738] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:40 2022] PUT /identity/v3/projects/34a63b45a42b43aab2c14072a2c2cc4a/users/8024f8899f5c48d6a59a66d9fb96c868/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:40.809799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:40.841276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84710013-14d0-4b8b-afb1-83b03d6644c0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:41.121406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a33aa-5db5-414a-b82f-f9557498bbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:41.122147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a33aa-5db5-414a-b82f-f9557498bbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:41.122386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a33aa-5db5-414a-b82f-f9557498bbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:41.122620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a33aa-5db5-414a-b82f-f9557498bbb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:41.123280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a33aa-5db5-414a-b82f-f9557498bbb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:41.124931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a33aa-5db5-414a-b82f-f9557498bbb4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:41.174376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 860/1739] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:41.175749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:41.195721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6d5b3-2b4e-4f0a-a3ed-20d6b58b8451 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:41.239087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:41.239482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:41.239765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:41.240051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] PATH_INFO: `/v3/projects/34a63b45a42b43aab2c14072a2c2cc4a/users/8024f8899f5c48d6a59a66d9fb96c868/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:41.251595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a63b45a42b43aab2c14072a2c2cc4a, user_id=8024f8899f5c48d6a59a66d9fb96c868, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:41.254294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:41.303044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 880/1740] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:41 2022] PUT /identity/v3/projects/34a63b45a42b43aab2c14072a2c2cc4a/users/8024f8899f5c48d6a59a66d9fb96c868/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:41.313602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171b7c4-fa97-4eaf-aa04-e3163ec8bc2c None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:41.314236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171b7c4-fa97-4eaf-aa04-e3163ec8bc2c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:41.314690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171b7c4-fa97-4eaf-aa04-e3163ec8bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:41.422385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2171b7c4-fa97-4eaf-aa04-e3163ec8bc2c None None] MFA Rules not processed for user `8024f8899f5c48d6a59a66d9fb96c868`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:41.423661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2171b7c4-fa97-4eaf-aa04-e3163ec8bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:41.673707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 861/1741] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:41.699481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:41.703224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:41.847209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaedcc-82b1-47db-a4c9-af6da5747e33 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:41.855015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3440a7-0b43-4adf-97bb-96bbcb43ab38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:41.855648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3440a7-0b43-4adf-97bb-96bbcb43ab38 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:41.856507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3440a7-0b43-4adf-97bb-96bbcb43ab38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:41.856985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3440a7-0b43-4adf-97bb-96bbcb43ab38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:41.894484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3440a7-0b43-4adf-97bb-96bbcb43ab38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:41.895649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3440a7-0b43-4adf-97bb-96bbcb43ab38 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:41.918367 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 881/1742] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:12:41.998828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171b7c4-fa97-4eaf-aa04-e3163ec8bc2c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:42.132551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171b7c4-fa97-4eaf-aa04-e3163ec8bc2c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:42.138651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:42.139389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:42.140334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:42.140643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] PATH_INFO: `/v3/users/164bd6fe13664a66a23935c610cbbfd0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:42.146301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=164bd6fe13664a66a23935c610cbbfd0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:42.148590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:42.168762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:42.283112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164bd6fe13664a66a23935c610cbbfd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:42.303668 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164bd6fe13664a66a23935c610cbbfd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:42.339468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164bd6fe13664a66a23935c610cbbfd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:42.340298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164bd6fe13664a66a23935c610cbbfd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:42.341167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164bd6fe13664a66a23935c610cbbfd0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:42.374259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 862/1743] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:41 2022] DELETE /identity/v3/users/164bd6fe13664a66a23935c610cbbfd0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:42.376744 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3fa5e-4224-4ff5-8708-2a1f91e35130 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:42.377103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3fa5e-4224-4ff5-8708-2a1f91e35130 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:42.377371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3fa5e-4224-4ff5-8708-2a1f91e35130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:42.429935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c9c3fa5e-4224-4ff5-8708-2a1f91e35130 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:42.431266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c9c3fa5e-4224-4ff5-8708-2a1f91e35130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:42.523753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:42.600855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 882/1744] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:42.766370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:42.768823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:43.040853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71823f5-cccd-4dc5-89a5-2f377b18f143 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:43.054029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6c7e-02d0-4578-8e9a-afb96e681a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:43.054669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6c7e-02d0-4578-8e9a-afb96e681a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:43.054950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6c7e-02d0-4578-8e9a-afb96e681a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:43.055235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6c7e-02d0-4578-8e9a-afb96e681a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:43.075664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6c7e-02d0-4578-8e9a-afb96e681a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:43.077127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6c7e-02d0-4578-8e9a-afb96e681a99 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:43.107588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 863/1745] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:12:44.207214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a10490-c5da-48cd-9bb8-308ca24497d8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:44.208486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a10490-c5da-48cd-9bb8-308ca24497d8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:44.208486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a10490-c5da-48cd-9bb8-308ca24497d8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:44.211044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 883/1746] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:44.319514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb1a4-c3a4-4bd4-8ae1-17eb43f15681 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:44.320117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb1a4-c3a4-4bd4-8ae1-17eb43f15681 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:44.320117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb1a4-c3a4-4bd4-8ae1-17eb43f15681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:44.366366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:44.366646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:44.367151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:44.417396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-68afb1a4-c3a4-4bd4-8ae1-17eb43f15681 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:44.427290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-68afb1a4-c3a4-4bd4-8ae1-17eb43f15681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:44.440323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] MFA Rules not processed for user `a22a06667e10411e8d8ce92db4536a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:44.443263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:44.618405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 884/1747] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:44.628908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:44.632546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:44.660310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 864/1748] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:12:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:44.687614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ffe44-ea82-4f44-b609-9c0a722b5ea0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:44.717871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:44.719043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:44.719497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:44.719953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] PATH_INFO: `/v3/projects/03e4a7ff8c8d4f63a07d13f565fe3226` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:44.726538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e4a7ff8c8d4f63a07d13f565fe3226)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:44.735158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:44.938238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e4a7ff8c8d4f63a07d13f565fe3226', 'request_id': 'req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:44.953689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e4a7ff8c8d4f63a07d13f565fe3226', 'request_id': 'req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:44.964564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e4a7ff8c8d4f63a07d13f565fe3226', 'request_id': 'req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:44.990632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] The token cache is being invalidate because project 03e4a7ff8c8d4f63a07d13f565fe3226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:44.990917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:44.991636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-794aed5f-60f9-4e1c-a4ca-6f83f6ed41d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:44.993784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 885/1749] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:44 2022] DELETE /identity/v3/projects/03e4a7ff8c8d4f63a07d13f565fe3226 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:45.007900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e27ed-d5fe-4db0-a607-e64e25ebdd8e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.008315 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e27ed-d5fe-4db0-a607-e64e25ebdd8e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.008701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e27ed-d5fe-4db0-a607-e64e25ebdd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.071848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-066e27ed-d5fe-4db0-a607-e64e25ebdd8e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:45.073145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-066e27ed-d5fe-4db0-a607-e64e25ebdd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:45.109089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:45.234769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cae7a-bb1c-47b5-8c67-8afeafda4d2c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.235294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 865/1750] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:45.235838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cae7a-bb1c-47b5-8c67-8afeafda4d2c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.236404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cae7a-bb1c-47b5-8c67-8afeafda4d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.302992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7c2cae7a-bb1c-47b5-8c67-8afeafda4d2c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:45.306537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7c2cae7a-bb1c-47b5-8c67-8afeafda4d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:45.339662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 886/1751] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:45.343573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e27ed-d5fe-4db0-a607-e64e25ebdd8e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:45.360555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e27ed-d5fe-4db0-a607-e64e25ebdd8e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:45.369750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:45.373128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.373499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.373906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.375801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:45.377007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:45.379974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:45.418244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 866/1752] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:12:45.419392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cae7a-bb1c-47b5-8c67-8afeafda4d2c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:45.432455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cae7a-bb1c-47b5-8c67-8afeafda4d2c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:45.443836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:45.447941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.448501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.448971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.451930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:45.455694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:45.519905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:45.530511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 887/1753] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:45 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:12:45.538640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e7463-5c9a-4265-a1b3-0189ca6d59dd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:45.681319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:45.682427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.683109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.683744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.684772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:45.689574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:45.748536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:45.752030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:45.760507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 867/1754] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:45 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:45.765336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:45.789634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84912d-7ebd-410e-872c-fc51d39580e8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:45.796320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724f072-1d86-4111-b6cd-ad60de749b93 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:45.804789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:45.805705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.806247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.806739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.807732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:45.811610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:45.822530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:45.823225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.823539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.823823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] PATH_INFO: `/v3/projects/2f376a605bc2430087314a29959cc86b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.825102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f376a605bc2430087314a29959cc86b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:45.827110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:45.848509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 868/1755] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:45.851170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:45.874644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be419ce3-feda-46fe-8acb-504d123fe012 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:45.887400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:45.890428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:45.890949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:45.893066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:45.895930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:45.901643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:45.909896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:45.930152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 869/1756] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:45.931352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f376a605bc2430087314a29959cc86b', 'request_id': 'req-38977327-75e2-485b-8f8a-d6badb3c381d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:45.934178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:45.954756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f376a605bc2430087314a29959cc86b', 'request_id': 'req-38977327-75e2-485b-8f8a-d6badb3c381d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:45.956367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5dfdb-8655-4f70-8fc9-745d783557e9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:45.982711 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f376a605bc2430087314a29959cc86b', 'request_id': 'req-38977327-75e2-485b-8f8a-d6badb3c381d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.041673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] The token cache is being invalidate because project 2f376a605bc2430087314a29959cc86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:46.044073 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.046787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.063261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 888/1757] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:45 2022] DELETE /identity/v3/projects/2f376a605bc2430087314a29959cc86b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:46.064297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:46.072927 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:46.121361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771340-d3c0-4b6f-8e67-67ff5a87ccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:46.125469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771340-d3c0-4b6f-8e67-67ff5a87ccc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:46.125786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771340-d3c0-4b6f-8e67-67ff5a87ccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:46.126069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771340-d3c0-4b6f-8e67-67ff5a87ccc7 admin admin] PATH_INFO: `/v3/projects/d687cfebc20a45f2abee056a41cc54ff/users/85b2e3c253df47c788f8495c0845e701/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:46.165176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77771340-d3c0-4b6f-8e67-67ff5a87ccc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d687cfebc20a45f2abee056a41cc54ff, user_id=85b2e3c253df47c788f8495c0845e701, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:46.175102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77771340-d3c0-4b6f-8e67-67ff5a87ccc7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:46.212290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:46.252048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 870/1758] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:45 2022] PUT /identity/v3/projects/d687cfebc20a45f2abee056a41cc54ff/users/85b2e3c253df47c788f8495c0845e701/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:46.253581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf167e-4e5f-4ef1-b8fc-15bcd135127c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:46.256498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf167e-4e5f-4ef1-b8fc-15bcd135127c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:46.256813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf167e-4e5f-4ef1-b8fc-15bcd135127c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:46.270796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 871/1759] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:12:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:12:46.270796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827081b-587c-483e-9ab0-193e5fd1fc24 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:46.270796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827081b-587c-483e-9ab0-193e5fd1fc24 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:46.270796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827081b-587c-483e-9ab0-193e5fd1fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:46.333401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38977327-75e2-485b-8f8a-d6badb3c381d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:46.372811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d827081b-587c-483e-9ab0-193e5fd1fc24 None None] MFA Rules not processed for user `85b2e3c253df47c788f8495c0845e701`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:46.374072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d827081b-587c-483e-9ab0-193e5fd1fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:46.514682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 872/1760] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:46.516315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:46.517332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:46.518121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:46.518424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:46.518831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] PATH_INFO: `/v3/users/8024f8899f5c48d6a59a66d9fb96c868` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:46.518950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:46.519333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:46.521705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=8024f8899f5c48d6a59a66d9fb96c868)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:46.530964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:46.587013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:46.588287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:46.678150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8024f8899f5c48d6a59a66d9fb96c868'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.736170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8024f8899f5c48d6a59a66d9fb96c868'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.754382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 873/1761] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:12:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:46.755793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:46.762333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:46.792992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086710a1-80e2-4590-b155-60b457ea2028 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:46.806489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:46.807173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:46.807451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:46.807742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:46.808561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:46.808989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:46.810976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:46.814284 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8024f8899f5c48d6a59a66d9fb96c868'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.834831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8024f8899f5c48d6a59a66d9fb96c868'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.836900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8024f8899f5c48d6a59a66d9fb96c868'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:46.855402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 889/1762] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:46 2022] DELETE /identity/v3/users/8024f8899f5c48d6a59a66d9fb96c868 => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:46.882605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 874/1763] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:46 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:46.884135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:46.886518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:47.018126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3dc27-5eb0-4805-8eb6-d1cbe673c0de admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:47.211879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:47.221987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:47.224559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:47.226247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:47.229323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:47.237657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:47.308346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 890/1764] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:46 2022] POST /identity/v3/users => generated 294 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:12:47.312476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:47.332318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a22a8-ca49-472c-b8ef-1a4a4adf4713 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:47.339997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:47.340628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:47.341258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:47.341258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:47.342007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:47.344044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:47.363451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 875/1765] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:12:47.368524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:47.381157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d20bed-e35f-473e-b24b-3bae4dc316b8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:47.403609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49674c6a-5b6b-4cf1-854a-4062526c9730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:47.406630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49674c6a-5b6b-4cf1-854a-4062526c9730 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:47.408128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49674c6a-5b6b-4cf1-854a-4062526c9730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:47.409039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49674c6a-5b6b-4cf1-854a-4062526c9730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:47.410735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49674c6a-5b6b-4cf1-854a-4062526c9730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:47.417684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49674c6a-5b6b-4cf1-854a-4062526c9730 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:47.448818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:47.462774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 891/1766] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:12:47.465956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17793a-b0da-4109-8c8e-67c8064b5b30 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:47.477748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:47.481013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:47.482384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:47.483674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] PATH_INFO: `/v3/projects/cd9a7996a31f4ccb9053566617c773c7/users/75d663b741114fb1ad2d5addc5b34963/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:47.563227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9a7996a31f4ccb9053566617c773c7, user_id=75d663b741114fb1ad2d5addc5b34963, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:47.571978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:47.628864 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 876/1767] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:47 2022] PUT /identity/v3/projects/cd9a7996a31f4ccb9053566617c773c7/users/75d663b741114fb1ad2d5addc5b34963/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:47.634669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d29f3c-0b67-43f9-b439-ca9869767a90 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:47.635411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d29f3c-0b67-43f9-b439-ca9869767a90 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:47.635915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d29f3c-0b67-43f9-b439-ca9869767a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:47.679341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-83d29f3c-0b67-43f9-b439-ca9869767a90 None None] MFA Rules not processed for user `75d663b741114fb1ad2d5addc5b34963`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:47.681206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-83d29f3c-0b67-43f9-b439-ca9869767a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:47.870339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 892/1768] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:47.872665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:47.931146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:47.995909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc06df-5c24-4447-a7ba-99afdcc5dbd2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:48.013417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:48.014117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.014404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.014830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.015664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:48.017936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:48.090791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 877/1769] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:47 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:48.092063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d29f3c-0b67-43f9-b439-ca9869767a90 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:48.105448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:48.110356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d29f3c-0b67-43f9-b439-ca9869767a90 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:48.285887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:48.286542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.286830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.287089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.287788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:48.306606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:48.352651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 893/1770] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:48 2022] POST /identity/v3/users => generated 300 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:48.353923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:48.367974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a966ea-6137-43cb-aaad-bca278ff2c49 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:48.380847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:48.381778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.382258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.385905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.386832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:48.388799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:48.426058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 878/1771] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:48.430822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:48.442549 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cf010-a290-4b53-88c1-afefe9addaac admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:48.462726 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:48.463399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.463678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.463949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] PATH_INFO: `/v3/projects/d0cd1e8158e04268aef1e0fee88b4e06/users/ec8ff4f4c4da4409ab09f4d5ca29802c/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.503730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cd1e8158e04268aef1e0fee88b4e06, user_id=ec8ff4f4c4da4409ab09f4d5ca29802c, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:48.509474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:48.565110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 894/1772] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:48 2022] PUT /identity/v3/projects/d0cd1e8158e04268aef1e0fee88b4e06/users/ec8ff4f4c4da4409ab09f4d5ca29802c/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:48.566476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:48.578971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ccd43-2dc9-4c5a-a86c-36d7e42245e8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:48.713940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:48.716283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:48.724406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:48.725026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.725303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.725568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.726435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:48.728248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:48.737769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2649b-54be-400b-9775-c25253c0ce14 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:48.754441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:48.755135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.755418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.755690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] PATH_INFO: `/v3/projects/34a63b45a42b43aab2c14072a2c2cc4a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.759212 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a63b45a42b43aab2c14072a2c2cc4a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:48.761052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:48.761886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 879/1773] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:48.763283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:48.782513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139804b-b9ee-407c-b972-6c4099fdb9e1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:48.799345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ed1ac-f0e7-4ca6-a60f-545dd2f60186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:48.799742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed1ac-f0e7-4ca6-a60f-545dd2f60186 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.800015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed1ac-f0e7-4ca6-a60f-545dd2f60186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.800286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed1ac-f0e7-4ca6-a60f-545dd2f60186 admin admin] PATH_INFO: `/v3/projects/d0cd1e8158e04268aef1e0fee88b4e06/users/ec8ff4f4c4da4409ab09f4d5ca29802c/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.803227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ed1ac-f0e7-4ca6-a60f-545dd2f60186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cd1e8158e04268aef1e0fee88b4e06, user_id=ec8ff4f4c4da4409ab09f4d5ca29802c, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:48.821644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a63b45a42b43aab2c14072a2c2cc4a', 'request_id': 'req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:48.823894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ed1ac-f0e7-4ca6-a60f-545dd2f60186 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:48.829496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a63b45a42b43aab2c14072a2c2cc4a', 'request_id': 'req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:48.840197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a63b45a42b43aab2c14072a2c2cc4a', 'request_id': 'req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:48.848755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 880/1774] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:48 2022] PUT /identity/v3/projects/d0cd1e8158e04268aef1e0fee88b4e06/users/ec8ff4f4c4da4409ab09f4d5ca29802c/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:48.860942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.861446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.862028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.894698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] The token cache is being invalidate because project 34a63b45a42b43aab2c14072a2c2cc4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:48.898444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:48.900016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a78e18a4-6ed0-48e6-9b45-c1e7f4ac7c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:48.903912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 895/1775] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:48 2022] DELETE /identity/v3/projects/34a63b45a42b43aab2c14072a2c2cc4a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:48.909518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4cc7-5d7c-4004-8b37-27f13804efff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:48.910033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4cc7-5d7c-4004-8b37-27f13804efff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:48.910453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4cc7-5d7c-4004-8b37-27f13804efff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:48.951643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] MFA Rules not processed for user `ec8ff4f4c4da4409ab09f4d5ca29802c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:48.953235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:48.965907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b0ed4cc7-5d7c-4004-8b37-27f13804efff None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:48.968942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0ed4cc7-5d7c-4004-8b37-27f13804efff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:49.060134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:49.090156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 896/1776] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:49.101141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939b53f-817b-4193-960b-7130a0d0df65 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.103047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939b53f-817b-4193-960b-7130a0d0df65 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.104482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939b53f-817b-4193-960b-7130a0d0df65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.107874 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 881/1777] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:49.123318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.126523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:49.135068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:49.158725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6939b53f-817b-4193-960b-7130a0d0df65 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:49.159911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6939b53f-817b-4193-960b-7130a0d0df65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:49.186721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 897/1778] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:49.188004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939b53f-817b-4193-960b-7130a0d0df65 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.197714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939b53f-817b-4193-960b-7130a0d0df65 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.204465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.205439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.205743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.206058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.206887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:12:49.208261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.209637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.224065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 898/1779] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:12:49.228592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.239614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32827f-67b0-4233-8a34-200f8223fd9c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.244915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.245664 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.246189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.246639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.247416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.249590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.254007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb4f70-eae7-4b02-a71d-2fe13b7aa712 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.263790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.266243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.266760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.268080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.293177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 899/1780] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:49 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:12:49.295448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.296692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.299067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.313339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 882/1781] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:12:49.375342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ff13-dcab-4401-b4c3-8468c26abd49 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.381443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.382289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.382686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.383057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.383795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.385896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.417795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 900/1782] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:49 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:49.419420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.433472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3e213-1031-468f-b8be-a48202e42c53 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.438749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.439558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.439922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.440445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.441198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.443070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.463590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 883/1783] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:49.464997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.477414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c9319-797d-4506-bcae-2db6e2ad90f4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.483409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd6028-1eb9-4b17-b826-778430f1345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.484496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6028-1eb9-4b17-b826-778430f1345c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.484880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6028-1eb9-4b17-b826-778430f1345c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.493081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd6028-1eb9-4b17-b826-778430f1345c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.493081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd6028-1eb9-4b17-b826-778430f1345c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.493081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd6028-1eb9-4b17-b826-778430f1345c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.504592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 901/1784] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:49.510491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.524483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef163-d2bf-41b4-aaf8-df11541b0a0b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.537243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.538359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.545210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.545893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] PATH_INFO: `/v3/projects/92eabc8312e6465e98d0a032dd04d1b0/users/1910cbab5e314f5bb0d96f26790139e4/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.573137 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92eabc8312e6465e98d0a032dd04d1b0, user_id=1910cbab5e314f5bb0d96f26790139e4, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.580428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.617612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 884/1785] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:49 2022] PUT /identity/v3/projects/92eabc8312e6465e98d0a032dd04d1b0/users/1910cbab5e314f5bb0d96f26790139e4/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:49.632332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5130cc-57ae-4cc2-be39-adc09b32cb44 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.632633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5130cc-57ae-4cc2-be39-adc09b32cb44 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.632944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5130cc-57ae-4cc2-be39-adc09b32cb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.674889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bf5130cc-57ae-4cc2-be39-adc09b32cb44 None None] MFA Rules not processed for user `1910cbab5e314f5bb0d96f26790139e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:49.676278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bf5130cc-57ae-4cc2-be39-adc09b32cb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:49.766733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 902/1786] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:49.773294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.858622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2742c0-0dc7-4bed-884f-32f440184604 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.864762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:49.865516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:49.865870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:49.866188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:49.866839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:49.869281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:49.929157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 885/1787] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:49 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:49.933065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5130cc-57ae-4cc2-be39-adc09b32cb44 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:49.946712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5130cc-57ae-4cc2-be39-adc09b32cb44 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:49.966001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:50.030415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.031170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.031739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.032124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.032943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:50.035737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:50.077000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 903/1788] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:49 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:50.086890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:50.095396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:50.103086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c7a9-0a78-43c9-b753-3e9b330ad05b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:50.113758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.114971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.116044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.116044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.118484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:50.120210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:50.141717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 886/1789] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:50.147248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:50.164432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89678fb-0345-4b34-b88b-0c1bd7503575 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:50.170846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.171570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.172042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.172501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.173275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:50.175544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:50.180981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724b3ce-acc1-44fd-9227-e7cd6d195df9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:50.186735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.187725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.188907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 887/1790] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:50.189206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.189490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.191317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:50.194017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:50.212792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658b502-cdce-44a6-a6dc-c7acae44fe49 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:50.221171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d704f-063e-4f87-92e3-e6557ae751df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.234937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d704f-063e-4f87-92e3-e6557ae751df admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.235457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d704f-063e-4f87-92e3-e6557ae751df admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.235888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d704f-063e-4f87-92e3-e6557ae751df admin admin] PATH_INFO: `/v3/projects/7f9e8450917b4fd59d960d6e1e3d3fc3/users/e00bf48268e94a568ec7c40da4d1ff7e/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.269470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d704f-063e-4f87-92e3-e6557ae751df admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9e8450917b4fd59d960d6e1e3d3fc3, user_id=e00bf48268e94a568ec7c40da4d1ff7e, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:50.277247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d704f-063e-4f87-92e3-e6557ae751df admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:50.304757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 888/1791] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:50 2022] PUT /identity/v3/projects/7f9e8450917b4fd59d960d6e1e3d3fc3/users/e00bf48268e94a568ec7c40da4d1ff7e/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:50.316711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614e57-217f-4701-b89f-39ae4dafe337 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.317394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614e57-217f-4701-b89f-39ae4dafe337 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.317842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614e57-217f-4701-b89f-39ae4dafe337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.346072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:50.367072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7d614e57-217f-4701-b89f-39ae4dafe337 None None] MFA Rules not processed for user `e00bf48268e94a568ec7c40da4d1ff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:50.368784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7d614e57-217f-4701-b89f-39ae4dafe337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:50.376908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:50.378430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:50.414061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 904/1792] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:50.518082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 889/1793] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:50.526950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:50.542364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64e7a0-76ef-403f-9368-1c2326bbe7c7 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:50.690406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.691199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.691568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.691850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.693944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:50.697243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:50.713013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:50.750439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 905/1794] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:12:50 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:12:50.750614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614e57-217f-4701-b89f-39ae4dafe337 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:50.764002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614e57-217f-4701-b89f-39ae4dafe337 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:50.901353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.901692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:50.902201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:50.902447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:50.904149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:50.906709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:50.950175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 890/1795] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:12:50 2022] POST /identity/v3/users => generated 295 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:12:50.951674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:50.972830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83c498-0c22-4d09-a829-74f8d67a3fb6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:50.984578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:50.999993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.003172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.003172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.005195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.009874 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:51.011583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:51.011983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.039682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 906/1796] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:51.046131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:51.095347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9f318-d0b9-4ca3-97ad-6e52a9b0fe00 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:51.118424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:51.119100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.119370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.119682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] PATH_INFO: `/v3/projects/aeafe68de69942159c95c6972f6f2d30/users/22200cb6c9a54d6494de4e6deb21450f/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.136826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e810522-6cf2-42b7-bd7d-e3a19108ba9a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:51.143690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:51.144417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.144698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.145139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.148532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:51.150998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeafe68de69942159c95c6972f6f2d30, user_id=22200cb6c9a54d6494de4e6deb21450f, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.154981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.179559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.181546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.194353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 907/1797] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:51 2022] PUT /identity/v3/projects/aeafe68de69942159c95c6972f6f2d30/users/22200cb6c9a54d6494de4e6deb21450f/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:51.195356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:51.200855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 891/1798] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:51.286193 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c7458-9028-4e93-942d-ba491b13f77b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:51.293290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e3aca-d46d-4b6f-86d4-12f636ae7309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:51.294065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e3aca-d46d-4b6f-86d4-12f636ae7309 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.294588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e3aca-d46d-4b6f-86d4-12f636ae7309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.294914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e3aca-d46d-4b6f-86d4-12f636ae7309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.295813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e3aca-d46d-4b6f-86d4-12f636ae7309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.298042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e3aca-d46d-4b6f-86d4-12f636ae7309 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.317041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 908/1799] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:12:51.318882 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:51.334223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016c9b-32ef-4b21-b126-15569149d58c service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:51.340121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:51.340887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.341235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.341581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] PATH_INFO: `/v3/projects/aeafe68de69942159c95c6972f6f2d30/users/22200cb6c9a54d6494de4e6deb21450f/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.343727 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeafe68de69942159c95c6972f6f2d30, user_id=22200cb6c9a54d6494de4e6deb21450f, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.346809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.361855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 892/1800] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:12:51 2022] PUT /identity/v3/projects/aeafe68de69942159c95c6972f6f2d30/users/22200cb6c9a54d6494de4e6deb21450f/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:51.373676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.374249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.374595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.395721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:51.409708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:51.452065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] MFA Rules not processed for user `22200cb6c9a54d6494de4e6deb21450f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:51.453342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:51.576801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7f57f-49b0-443f-96ba-057432da0cc9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:51.586263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:51.586896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.587317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.587599 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.589962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:51.612057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 909/1801] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:51.623055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.625070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.627392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:51.631089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:51.640947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 893/1802] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:51.702741 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:51.748363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869a7f4-be02-4d00-a1ac-d1db477f8287 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:51.759389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:51.761761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.762242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.762679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.788371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.791544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.826410 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 910/1803] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:51.875511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:51.893445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5000e4-8d3d-4cad-9c69-42777103724a service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:51.900001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:51.900702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:51.900983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:51.901288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:51.967558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:51.969158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:51.979578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 894/1804] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:12:54.915620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:54.928111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46779109-3f77-4a3f-897b-0c15f2303183 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:54.932176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f8915-3404-4ed1-9ba1-c3c5459a35d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:54.933639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8915-3404-4ed1-9ba1-c3c5459a35d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:54.934080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8915-3404-4ed1-9ba1-c3c5459a35d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:54.935967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8915-3404-4ed1-9ba1-c3c5459a35d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:54.936990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-626f8915-3404-4ed1-9ba1-c3c5459a35d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:55.068098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f8915-3404-4ed1-9ba1-c3c5459a35d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:55.072892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f8915-3404-4ed1-9ba1-c3c5459a35d7 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:55.098950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 911/1805] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:55.315396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:55.538788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e640d3-f7cf-4191-8891-23739b5d92d2 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:55.544177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:55.544841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:55.545122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:55.545522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] PATH_INFO: `/v3/users/1910cbab5e314f5bb0d96f26790139e4` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:55.546794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=1910cbab5e314f5bb0d96f26790139e4)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:55.548938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:55.615492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:55.665958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1910cbab5e314f5bb0d96f26790139e4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:55.674380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1910cbab5e314f5bb0d96f26790139e4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:55.701072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1910cbab5e314f5bb0d96f26790139e4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:55.704464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1910cbab5e314f5bb0d96f26790139e4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:55.705693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1910cbab5e314f5bb0d96f26790139e4'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:55.729031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 895/1806] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:55 2022] DELETE /identity/v3/users/1910cbab5e314f5bb0d96f26790139e4 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:55.734149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e0115-9e0f-4ca0-88bb-d5c55aa62cd3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:55.734149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e0115-9e0f-4ca0-88bb-d5c55aa62cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:55.734643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e0115-9e0f-4ca0-88bb-d5c55aa62cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:55.758579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3f9e0115-9e0f-4ca0-88bb-d5c55aa62cd3 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:55.759961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f9e0115-9e0f-4ca0-88bb-d5c55aa62cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:55.894757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 912/1807] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:55.936358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:55.938441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:56.079462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9f941-55cb-4098-9110-ea92cbd349fe admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:56.093264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47c4f3-8e58-444b-91ce-b670933638da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:56.094103 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47c4f3-8e58-444b-91ce-b670933638da service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:56.094494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47c4f3-8e58-444b-91ce-b670933638da service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:56.094878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47c4f3-8e58-444b-91ce-b670933638da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:56.109959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47c4f3-8e58-444b-91ce-b670933638da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:56.111695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47c4f3-8e58-444b-91ce-b670933638da service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:56.140530 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 896/1808] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:56.380461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e0415-f2d5-4adf-9cdb-8fd7c1a0975f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:56.380919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e0415-f2d5-4adf-9cdb-8fd7c1a0975f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:56.380919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e0415-f2d5-4adf-9cdb-8fd7c1a0975f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:56.382327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 913/1809] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:12:56.400709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:56.401293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:56.401293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:56.451804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:56.452996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:56.700574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 897/1810] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:12:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:12:56.861314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e0415-f2d5-4adf-9cdb-8fd7c1a0975f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:56.864180 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1e8e0415-f2d5-4adf-9cdb-8fd7c1a0975f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:56.887759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e0415-f2d5-4adf-9cdb-8fd7c1a0975f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:56.894287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:56.895028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:56.899293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:56.899611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:56.912670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:56.914780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:56.974413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 914/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:57.248619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:57.264142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:57.288562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9458f-49e2-49f8-9d4f-17cfbc1f032a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:57.325048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:57.325756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:57.326441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:57.331274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] PATH_INFO: `/v3/projects/92eabc8312e6465e98d0a032dd04d1b0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:57.333294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] RBAC: Authorizing `identity:delete_project(project_id=92eabc8312e6465e98d0a032dd04d1b0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:57.341121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:57.414124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:57.453521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92eabc8312e6465e98d0a032dd04d1b0', 'request_id': 'req-963f9395-8c59-446e-8223-4b140f65a22d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.490200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92eabc8312e6465e98d0a032dd04d1b0', 'request_id': 'req-963f9395-8c59-446e-8223-4b140f65a22d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.506708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:57.508219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:57.511915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92eabc8312e6465e98d0a032dd04d1b0', 'request_id': 'req-963f9395-8c59-446e-8223-4b140f65a22d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.534682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] The token cache is being invalidate because project 92eabc8312e6465e98d0a032dd04d1b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:57.534990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.539269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.540380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8adec-f148-49be-b07f-5406bd77b075 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:57.545948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 898/1812] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:57 2022] DELETE /identity/v3/projects/92eabc8312e6465e98d0a032dd04d1b0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:57.547230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:57.548807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:57.625289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:57.626009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:57.626344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:57.626624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] PATH_INFO: `/v3/users/85b2e3c253df47c788f8495c0845e701` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:57.628187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b2e3c253df47c788f8495c0845e701)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:57.631247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:57.638494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9395-8c59-446e-8223-4b140f65a22d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:57.645330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:57.646671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:57.647436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:57.647620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] PATH_INFO: `/v3/users/e00bf48268e94a568ec7c40da4d1ff7e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:57.656982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:57.662094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00bf48268e94a568ec7c40da4d1ff7e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:57.665317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:57.746622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00bf48268e94a568ec7c40da4d1ff7e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.757196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00bf48268e94a568ec7c40da4d1ff7e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.777060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00bf48268e94a568ec7c40da4d1ff7e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.777906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00bf48268e94a568ec7c40da4d1ff7e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.778930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-86d218a0-d586-4ef5-af43-49a0c2c88c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00bf48268e94a568ec7c40da4d1ff7e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.787691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b2e3c253df47c788f8495c0845e701'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.795674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b2e3c253df47c788f8495c0845e701'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.820123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b2e3c253df47c788f8495c0845e701'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.821507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 899/1813] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:57 2022] DELETE /identity/v3/users/e00bf48268e94a568ec7c40da4d1ff7e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:57.834599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b2e3c253df47c788f8495c0845e701'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.835486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b2e3c253df47c788f8495c0845e701'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:57.844042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 915/1814] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:57 2022] DELETE /identity/v3/users/85b2e3c253df47c788f8495c0845e701 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:57.849980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:57.850262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:57.850543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:57.875173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:12:57.876574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:57.976858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:58.034182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 900/1815] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:12:58.046197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:58.047676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:58.182059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f74642-be93-4db4-b4c2-476cbb9e5b4d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:58.189889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:58.190841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:58.191218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:58.192436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:58.218730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:58.221336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:58.237542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 916/1816] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:12:58.990138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:58.991597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:59.004528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab538e32-cea1-4b10-a54b-414d1fa29877 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:59.022496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:59.023502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:59.023994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:59.024495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] PATH_INFO: `/v3/projects/7f9e8450917b4fd59d960d6e1e3d3fc3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:59.039570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9e8450917b4fd59d960d6e1e3d3fc3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:59.046326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:59.143669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9e8450917b4fd59d960d6e1e3d3fc3', 'request_id': 'req-f9e99f18-860d-48ef-9ccc-e7b7c1744072'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.156208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9e8450917b4fd59d960d6e1e3d3fc3', 'request_id': 'req-f9e99f18-860d-48ef-9ccc-e7b7c1744072'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.184691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9e8450917b4fd59d960d6e1e3d3fc3', 'request_id': 'req-f9e99f18-860d-48ef-9ccc-e7b7c1744072'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.199788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] The token cache is being invalidate because project 7f9e8450917b4fd59d960d6e1e3d3fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:59.200094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.200762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.207993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 901/1817] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:58 2022] DELETE /identity/v3/projects/7f9e8450917b4fd59d960d6e1e3d3fc3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:59.212285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:59.228097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cdb68-cc5c-4db6-90e9-bdfcd03a09e4 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:59.319697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:59.320576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:59.320866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:59.321141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] PATH_INFO: `/v3/users/22200cb6c9a54d6494de4e6deb21450f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:59.322894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22200cb6c9a54d6494de4e6deb21450f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:59.325315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:59.366006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:12:59.377355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:59.378848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:59.397494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e99f18-860d-48ef-9ccc-e7b7c1744072 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:59.404256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:59.405073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:59.405365 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:59.405645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] PATH_INFO: `/v3/projects/d687cfebc20a45f2abee056a41cc54ff` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:59.407472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d687cfebc20a45f2abee056a41cc54ff)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:59.409693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:59.415793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22200cb6c9a54d6494de4e6deb21450f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.432412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22200cb6c9a54d6494de4e6deb21450f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.476676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22200cb6c9a54d6494de4e6deb21450f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.493713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22200cb6c9a54d6494de4e6deb21450f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.494316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22200cb6c9a54d6494de4e6deb21450f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.534223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd687cfebc20a45f2abee056a41cc54ff', 'request_id': 'req-ef21cd74-db03-481b-86a3-07fde942ae91'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.535165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 917/1818] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:59 2022] DELETE /identity/v3/users/22200cb6c9a54d6494de4e6deb21450f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:12:59.543141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd687cfebc20a45f2abee056a41cc54ff', 'request_id': 'req-ef21cd74-db03-481b-86a3-07fde942ae91'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.554473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd687cfebc20a45f2abee056a41cc54ff', 'request_id': 'req-ef21cd74-db03-481b-86a3-07fde942ae91'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.574222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] The token cache is being invalidate because project d687cfebc20a45f2abee056a41cc54ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:12:59.575133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.576324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.578524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 902/1819] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:12:59 2022] DELETE /identity/v3/projects/d687cfebc20a45f2abee056a41cc54ff => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:12:59.583176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:12:59.584749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:12:59.693640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d1df6-6830-4533-8e9d-2f0d9459f787 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:12:59.699502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:12:59.700320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:12:59.700682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:12:59.701026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] PATH_INFO: `/v3/users/75d663b741114fb1ad2d5addc5b34963` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:12:59.702487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d663b741114fb1ad2d5addc5b34963)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:12:59.705291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:12:59.767599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d663b741114fb1ad2d5addc5b34963'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.778092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d663b741114fb1ad2d5addc5b34963'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.795859 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d663b741114fb1ad2d5addc5b34963'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.811230 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d663b741114fb1ad2d5addc5b34963'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.814800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d663b741114fb1ad2d5addc5b34963'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:12:59.824335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 918/1820] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:12:59 2022] DELETE /identity/v3/users/75d663b741114fb1ad2d5addc5b34963 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:00.275638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:00.277297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:00.296395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:00.297991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:00.358178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:00.378594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:00.407109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21cd74-db03-481b-86a3-07fde942ae91 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:00.477566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065928a-82aa-4a2c-b1ca-293a9a52a27a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:00.485269 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:00.486658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:00.487308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:00.487992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] PATH_INFO: `/v3/projects/aeafe68de69942159c95c6972f6f2d30` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:00.490494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeafe68de69942159c95c6972f6f2d30)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:00.493344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:00.552821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:00.553477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:00.553774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:00.554053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] PATH_INFO: `/v3/projects/cd9a7996a31f4ccb9053566617c773c7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:00.558817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9a7996a31f4ccb9053566617c773c7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:00.560955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:00.608918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeafe68de69942159c95c6972f6f2d30', 'request_id': 'req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.622035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeafe68de69942159c95c6972f6f2d30', 'request_id': 'req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.632790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeafe68de69942159c95c6972f6f2d30', 'request_id': 'req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.651780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9a7996a31f4ccb9053566617c773c7', 'request_id': 'req-0763750d-9c06-4839-8155-ac6da278a39c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.653353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] The token cache is being invalidate because project aeafe68de69942159c95c6972f6f2d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:00.653702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.654411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77d3b4be-3512-46e7-b6e2-de4e3ecbe3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.656109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 919/1821] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:00 2022] DELETE /identity/v3/projects/aeafe68de69942159c95c6972f6f2d30 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:00.663995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493032d1-2287-4295-9536-29c678c9db3b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:00.664431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493032d1-2287-4295-9536-29c678c9db3b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:00.664812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493032d1-2287-4295-9536-29c678c9db3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:00.676873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9a7996a31f4ccb9053566617c773c7', 'request_id': 'req-0763750d-9c06-4839-8155-ac6da278a39c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.696721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9a7996a31f4ccb9053566617c773c7', 'request_id': 'req-0763750d-9c06-4839-8155-ac6da278a39c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.697344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-493032d1-2287-4295-9536-29c678c9db3b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:00.698944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-493032d1-2287-4295-9536-29c678c9db3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:00.713537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] The token cache is being invalidate because project cd9a7996a31f4ccb9053566617c773c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:00.713995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.715388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:00.716602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 903/1822] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:00 2022] DELETE /identity/v3/projects/cd9a7996a31f4ccb9053566617c773c7 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:00.720501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:00.721932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:00.837455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 920/1823] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:00.839167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b9b2d-7ae7-4c09-95e8-70aec7bda297 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:00.839454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b9b2d-7ae7-4c09-95e8-70aec7bda297 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:00.839729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b9b2d-7ae7-4c09-95e8-70aec7bda297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:00.881659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-454b9b2d-7ae7-4c09-95e8-70aec7bda297 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:00.884533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-454b9b2d-7ae7-4c09-95e8-70aec7bda297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:00.904189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 921/1824] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:00.911573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b9b2d-7ae7-4c09-95e8-70aec7bda297 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:00.940594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763750d-9c06-4839-8155-ac6da278a39c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:00.948478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b9b2d-7ae7-4c09-95e8-70aec7bda297 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:00.958086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:00.958846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:00.959643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:00.959643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] PATH_INFO: `/v3/users/ec8ff4f4c4da4409ab09f4d5ca29802c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:00.964810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:00.965438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:00.965719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:00.966013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:00.966669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:00.966970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:00.968575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:00.975325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec8ff4f4c4da4409ab09f4d5ca29802c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:00.978540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:01.030635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 922/1825] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:01.030635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:01.065266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf3405-1aef-438a-b100-4c6e1ca75dbe admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:01.074355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:01.087643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:01.088287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:01.089930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:01.091295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:01.092375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:01.099039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:01.117267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec8ff4f4c4da4409ab09f4d5ca29802c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:01.133370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec8ff4f4c4da4409ab09f4d5ca29802c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:01.146126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 923/1826] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:01 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:01.150750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:01.173710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec8ff4f4c4da4409ab09f4d5ca29802c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:01.174868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec8ff4f4c4da4409ab09f4d5ca29802c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:01.175711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec8ff4f4c4da4409ab09f4d5ca29802c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:01.191820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:01.210083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 904/1827] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:00 2022] DELETE /identity/v3/users/ec8ff4f4c4da4409ab09f4d5ca29802c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:01.302006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac631de-11df-4a48-8386-17f3642d8c1c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:01.394514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:01.395202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:01.395671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:01.398653 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:01.399337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:01.401347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:01.428046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 924/1828] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:01 2022] POST /identity/v3/users => generated 288 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:01.431472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:01.434145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:01.453547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26703ee-eea7-4ad8-8376-fa42be469086 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:01.460567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:01.462206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:01.463013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:01.463826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:01.465118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:01.469046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:01.498043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 905/1829] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:01.499314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:01.517083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac9e6b-b65b-4d23-965b-832aa965ba69 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:01.524578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ce6d6-1e53-4951-a8be-491dad9062ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:01.525545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ce6d6-1e53-4951-a8be-491dad9062ac admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:01.525975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ce6d6-1e53-4951-a8be-491dad9062ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:01.526351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ce6d6-1e53-4951-a8be-491dad9062ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:01.527117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ce6d6-1e53-4951-a8be-491dad9062ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:01.529299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ce6d6-1e53-4951-a8be-491dad9062ac admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:01.553252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:01.561098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 925/1830] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:01.567690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fade24-fda5-41da-97a3-57582162fa1f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:01.575355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:01.576159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:01.576543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:01.576921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] PATH_INFO: `/v3/projects/24c3882ed5ac499f9aba5de4d43bf336/users/606e27c7d61546e1b546bab7109157fd/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:01.609887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c3882ed5ac499f9aba5de4d43bf336, user_id=606e27c7d61546e1b546bab7109157fd, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:01.614345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:01.635403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 906/1831] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:01 2022] PUT /identity/v3/projects/24c3882ed5ac499f9aba5de4d43bf336/users/606e27c7d61546e1b546bab7109157fd/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:01.648726 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:01.649347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:01.649531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:01.681885 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] MFA Rules not processed for user `606e27c7d61546e1b546bab7109157fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:01.683224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:01.772651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 926/1832] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:01.787817 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:01.790417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:01.872872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5933cb-3b80-4bc9-9830-44ac1b4a971d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:01.876982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31872ff-29c4-47b4-be37-9dbcd23d75b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:01.877604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31872ff-29c4-47b4-be37-9dbcd23d75b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:01.877924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31872ff-29c4-47b4-be37-9dbcd23d75b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:01.878198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31872ff-29c4-47b4-be37-9dbcd23d75b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:01.893991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31872ff-29c4-47b4-be37-9dbcd23d75b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:01.895861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31872ff-29c4-47b4-be37-9dbcd23d75b5 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:01.909913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 907/1833] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:01.991044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:01.992620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:02.054830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:02.130702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c2237-d63f-4fa8-b0d4-82427a54e693 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.135715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.136445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.136926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.137408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] PATH_INFO: `/v3/projects/d0cd1e8158e04268aef1e0fee88b4e06` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.138679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0cd1e8158e04268aef1e0fee88b4e06)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.140760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:02.220641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0cd1e8158e04268aef1e0fee88b4e06', 'request_id': 'req-fbba8f1e-9c46-4b82-afee-3c4166648fbf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:02.238335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0cd1e8158e04268aef1e0fee88b4e06', 'request_id': 'req-fbba8f1e-9c46-4b82-afee-3c4166648fbf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:02.240055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0cd1e8158e04268aef1e0fee88b4e06', 'request_id': 'req-fbba8f1e-9c46-4b82-afee-3c4166648fbf'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:02.260436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] The token cache is being invalidate because project d0cd1e8158e04268aef1e0fee88b4e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:02.260723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:02.261352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fbba8f1e-9c46-4b82-afee-3c4166648fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:02.265938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 927/1834] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:01 2022] DELETE /identity/v3/projects/d0cd1e8158e04268aef1e0fee88b4e06 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:02.269793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fab09-f922-4b72-a188-4c68970e26db None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.270134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fab09-f922-4b72-a188-4c68970e26db None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.270416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fab09-f922-4b72-a188-4c68970e26db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.299500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-980fab09-f922-4b72-a188-4c68970e26db None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:02.301721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-980fab09-f922-4b72-a188-4c68970e26db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:02.365195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 908/1835] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:02.373498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60502-07ac-4d23-babd-c066313d8eb1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.374007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60502-07ac-4d23-babd-c066313d8eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.374183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc60502-07ac-4d23-babd-c066313d8eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.395730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7cc60502-07ac-4d23-babd-c066313d8eb1 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:02.397174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7cc60502-07ac-4d23-babd-c066313d8eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:02.405477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 928/1836] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:02.409845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fab09-f922-4b72-a188-4c68970e26db None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:02.420222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fab09-f922-4b72-a188-4c68970e26db None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.424851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.425450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.425778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.426720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.427373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:02.427679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.429259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:02.438285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:02.445346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 909/1837] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:02.446270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc60502-07ac-4d23-babd-c066313d8eb1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:02.455652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc60502-07ac-4d23-babd-c066313d8eb1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.460389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.460981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.461261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.461536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.462223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.464103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:02.497893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 929/1838] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:02 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:02.498816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:02.511930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c456d-fa81-4a54-8f69-da1166a476ce admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.577508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.578131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.578411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.578683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.579311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.581223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:02.608408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 910/1839] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:02 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:02.609617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:02.620097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad102c1-c164-49b6-960d-8353e82d5453 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.624936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.625621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.625933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.626207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.627002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.628591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:02.642798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 930/1840] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:02.643657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:02.654653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654a790-a939-4275-93af-0cd5682bdb9b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.659591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7a209-40ed-4c42-bb06-e3d91243d8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.660241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7a209-40ed-4c42-bb06-e3d91243d8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.660617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7a209-40ed-4c42-bb06-e3d91243d8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.660927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7a209-40ed-4c42-bb06-e3d91243d8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.661575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7a209-40ed-4c42-bb06-e3d91243d8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.663222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7a209-40ed-4c42-bb06-e3d91243d8b8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:02.678730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 911/1841] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:02.679883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:02.690327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415eac4-644b-48ef-bbe9-2e3a5c94a1a6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.695780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.696448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.696725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.697103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] PATH_INFO: `/v3/projects/b9918b628b58490ca48de350a69bdd1c/users/88708a89b6cb4e97bbe40c9ba5b0bd40/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.713110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9918b628b58490ca48de350a69bdd1c, user_id=88708a89b6cb4e97bbe40c9ba5b0bd40, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.715787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:02.737536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 931/1842] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:02 2022] PUT /identity/v3/projects/b9918b628b58490ca48de350a69bdd1c/users/88708a89b6cb4e97bbe40c9ba5b0bd40/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:02.745760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be7405-22f6-4f0a-85b1-050a135e3443 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.746048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be7405-22f6-4f0a-85b1-050a135e3443 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.746324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be7405-22f6-4f0a-85b1-050a135e3443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.781290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-70be7405-22f6-4f0a-85b1-050a135e3443 None None] MFA Rules not processed for user `88708a89b6cb4e97bbe40c9ba5b0bd40`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:02.782693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-70be7405-22f6-4f0a-85b1-050a135e3443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:02.897870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 912/1843] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:02.899639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:02.966368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:02.971566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bf196-f0df-4308-861d-cb804806b33a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:02.990236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:02.991057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:02.991489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:02.991917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:02.992712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:02.994804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:03.027304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 932/1844] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:02 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:03.031117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be7405-22f6-4f0a-85b1-050a135e3443 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:03.043909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be7405-22f6-4f0a-85b1-050a135e3443 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:03.129497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:03.130154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.130728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.131096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.131759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:03.133678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:03.155805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:03.172784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 913/1845] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:03 2022] POST /identity/v3/users => generated 306 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:03.177363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:03.188857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f3d5-a0af-496e-b5c6-376b42b6de74 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:03.194493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:03.195484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.195947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.196378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.197177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:03.198981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:03.211964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 933/1846] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:03.216803 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:03.233643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e02dc-ea6f-4a50-a68c-3fc4c6b323b2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:03.240554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbc6eb-6dcc-463f-a703-fb34264d9c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:03.241546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbc6eb-6dcc-463f-a703-fb34264d9c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.242007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbc6eb-6dcc-463f-a703-fb34264d9c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.242450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbc6eb-6dcc-463f-a703-fb34264d9c30 admin admin] PATH_INFO: `/v3/projects/02c93ff3157b44599df7e4c0ad677d5e/users/3615bd2eb12340598e11e2f8d5edb5fc/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.261446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbc6eb-6dcc-463f-a703-fb34264d9c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c93ff3157b44599df7e4c0ad677d5e, user_id=3615bd2eb12340598e11e2f8d5edb5fc, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:03.264264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbc6eb-6dcc-463f-a703-fb34264d9c30 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:03.285622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 914/1847] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:03 2022] PUT /identity/v3/projects/02c93ff3157b44599df7e4c0ad677d5e/users/3615bd2eb12340598e11e2f8d5edb5fc/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:03.290001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:03.309559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ada795-9623-4da0-a3b3-0e2a6d6362e9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:03.408269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.408794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.409283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.436477 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] MFA Rules not processed for user `606e27c7d61546e1b546bab7109157fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:03.438671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:03.456291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:03.457841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.458162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.458637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.459299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:03.461661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:03.472385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 934/1848] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:03.478623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:03.498196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7c0c3-d00c-43a1-aa52-16c593f1ecec admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:03.516025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b092746-9d8f-4e89-b9c9-a9fcc648a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:03.516773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b092746-9d8f-4e89-b9c9-a9fcc648a88d admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.517108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b092746-9d8f-4e89-b9c9-a9fcc648a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.517387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b092746-9d8f-4e89-b9c9-a9fcc648a88d admin admin] PATH_INFO: `/v3/projects/02c93ff3157b44599df7e4c0ad677d5e/users/3615bd2eb12340598e11e2f8d5edb5fc/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.520363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b092746-9d8f-4e89-b9c9-a9fcc648a88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c93ff3157b44599df7e4c0ad677d5e, user_id=3615bd2eb12340598e11e2f8d5edb5fc, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:03.524318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b092746-9d8f-4e89-b9c9-a9fcc648a88d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:03.546309 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 935/1849] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:03 2022] PUT /identity/v3/projects/02c93ff3157b44599df7e4c0ad677d5e/users/3615bd2eb12340598e11e2f8d5edb5fc/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:03.556079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 915/1850] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:03.560642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381c0ea-ce60-4aaf-bcdb-20173b5684ec None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.561019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381c0ea-ce60-4aaf-bcdb-20173b5684ec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.561224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381c0ea-ce60-4aaf-bcdb-20173b5684ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.593213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2381c0ea-ce60-4aaf-bcdb-20173b5684ec None None] MFA Rules not processed for user `3615bd2eb12340598e11e2f8d5edb5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:03.594564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2381c0ea-ce60-4aaf-bcdb-20173b5684ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:03.683133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 936/1851] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:03.694167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:03.695589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:03.753225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17969ca0-b228-4c70-a544-791a16c68115 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:03.803202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:03.803809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.804085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.804358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.813670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:03.815130 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:03.824814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 916/1852] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:03.918759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381c0ea-ce60-4aaf-bcdb-20173b5684ec None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:03.928737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381c0ea-ce60-4aaf-bcdb-20173b5684ec None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:03.932529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a535bb-848b-456a-a9e8-c894f36fc841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:03.933274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a535bb-848b-456a-a9e8-c894f36fc841 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:03.933615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a535bb-848b-456a-a9e8-c894f36fc841 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:03.934135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a535bb-848b-456a-a9e8-c894f36fc841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:03.978816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:04.009277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a535bb-848b-456a-a9e8-c894f36fc841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:04.010750 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a535bb-848b-456a-a9e8-c894f36fc841 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:04.021584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 937/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:08.463938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:08.543086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:08.604683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda3d9e-bca8-4951-ac6b-bcb70a6235c0 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:08.609962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:08.611377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:08.611766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:08.612151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] PATH_INFO: `/v3/users/88708a89b6cb4e97bbe40c9ba5b0bd40` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:08.614095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88708a89b6cb4e97bbe40c9ba5b0bd40)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:08.617656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:08.684392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88708a89b6cb4e97bbe40c9ba5b0bd40'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:08.694317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88708a89b6cb4e97bbe40c9ba5b0bd40'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:08.736971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88708a89b6cb4e97bbe40c9ba5b0bd40'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:08.740283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88708a89b6cb4e97bbe40c9ba5b0bd40'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:08.740925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88708a89b6cb4e97bbe40c9ba5b0bd40'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:08.764680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 917/1854] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:08 2022] DELETE /identity/v3/users/88708a89b6cb4e97bbe40c9ba5b0bd40 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:08.766599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af262a33-504a-4e90-88a6-854a369df48c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:08.767136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af262a33-504a-4e90-88a6-854a369df48c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:08.767531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af262a33-504a-4e90-88a6-854a369df48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:08.793789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-af262a33-504a-4e90-88a6-854a369df48c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:08.795238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-af262a33-504a-4e90-88a6-854a369df48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:08.908003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 938/1855] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:08.967161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:08.969855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:09.091113 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f21ce4-f607-48c9-a1b0-fb4ab0024483 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:09.107843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:09.108480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:09.108761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:09.109035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:09.127084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:09.130735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:09.142862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 918/1856] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:10.668159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af262a33-504a-4e90-88a6-854a369df48c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:10.670492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-af262a33-504a-4e90-88a6-854a369df48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:10.684758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af262a33-504a-4e90-88a6-854a369df48c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:10.689767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:10.690595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:10.691138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:10.691516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] PATH_INFO: `/v3/projects/b9918b628b58490ca48de350a69bdd1c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:10.692781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9918b628b58490ca48de350a69bdd1c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:10.695094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:10.761008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9918b628b58490ca48de350a69bdd1c', 'request_id': 'req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:10.772849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9918b628b58490ca48de350a69bdd1c', 'request_id': 'req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:10.781637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9918b628b58490ca48de350a69bdd1c', 'request_id': 'req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:10.819034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] The token cache is being invalidate because project b9918b628b58490ca48de350a69bdd1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:10.819462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:10.822736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:10.827447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 939/1857] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:10 2022] DELETE /identity/v3/projects/b9918b628b58490ca48de350a69bdd1c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:10.829031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:10.843996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61564382-ebc5-4faf-88f6-ed63a9aa0d58 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:10.967845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:10.968498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:10.968935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:10.969487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] PATH_INFO: `/v3/users/3615bd2eb12340598e11e2f8d5edb5fc` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:10.971025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3615bd2eb12340598e11e2f8d5edb5fc)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:10.973974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:10.975755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:11.065400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3615bd2eb12340598e11e2f8d5edb5fc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:11.087196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3615bd2eb12340598e11e2f8d5edb5fc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:11.111406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3615bd2eb12340598e11e2f8d5edb5fc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:11.112197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3615bd2eb12340598e11e2f8d5edb5fc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:11.112829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-314a8481-805f-411d-8fca-57f89456ff89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3615bd2eb12340598e11e2f8d5edb5fc'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:11.133288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 919/1858] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:10 2022] DELETE /identity/v3/users/3615bd2eb12340598e11e2f8d5edb5fc => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:12.565172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:12.567660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:12.598587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d6ed1d-dbb3-4a16-9b17-bccf4b9c79ff admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:12.786306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:12.786963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:12.787248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:12.787522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] PATH_INFO: `/v3/projects/02c93ff3157b44599df7e4c0ad677d5e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:12.792000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c93ff3157b44599df7e4c0ad677d5e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:12.792858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:12.803241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:12.917293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c93ff3157b44599df7e4c0ad677d5e', 'request_id': 'req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:12.935131 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c93ff3157b44599df7e4c0ad677d5e', 'request_id': 'req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:12.951034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c93ff3157b44599df7e4c0ad677d5e', 'request_id': 'req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:12.978286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] The token cache is being invalidate because project 02c93ff3157b44599df7e4c0ad677d5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:12.978594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:12.983501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1ff84d6c-72bd-4cf6-b541-5393139b1a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:12.992582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 940/1859] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:12 2022] DELETE /identity/v3/projects/02c93ff3157b44599df7e4c0ad677d5e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:12.997403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be472-ed92-4eae-b827-7d277ed03cf6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:12.998475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be472-ed92-4eae-b827-7d277ed03cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:12.998972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6be472-ed92-4eae-b827-7d277ed03cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.035994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1d6be472-ed92-4eae-b827-7d277ed03cf6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:13.040974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1d6be472-ed92-4eae-b827-7d277ed03cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:13.167285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 920/1860] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:13.176335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08fcc4-c336-49e5-a139-139769b95841 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.176975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08fcc4-c336-49e5-a139-139769b95841 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.177547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08fcc4-c336-49e5-a139-139769b95841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.237113 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7d08fcc4-c336-49e5-a139-139769b95841 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:13.239516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7d08fcc4-c336-49e5-a139-139769b95841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:13.248730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 941/1861] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:13.252645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6be472-ed92-4eae-b827-7d277ed03cf6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:13.267625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6be472-ed92-4eae-b827-7d277ed03cf6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:13.298351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:13.299577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.301276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.302288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.304799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:13.305475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:13.310517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:13.350338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 921/1862] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:13.369690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08fcc4-c336-49e5-a139-139769b95841 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:13.381322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08fcc4-c336-49e5-a139-139769b95841 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:13.387956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:13.388805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.389248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.389671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.390909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:13.393682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:13.444377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 942/1863] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:13 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:13.448642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:13.469253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3c71b-d3df-492f-b835-aff6bd72e602 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:13.599382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:13.600845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.601400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.601688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.602443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:13.604865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:13.632063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:13.655829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 922/1864] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:13 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:13.660791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:13.690719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7d40-806e-4c6e-9479-1446a53f0d00 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:13.705551 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:13.706232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.706518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.706831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.707875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:13.709641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:13.735400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 943/1865] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:13.741327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:13.781653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb623-fe9c-4ee8-9dd8-71b26f30f9e8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:13.789554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dbe5b-75bc-4c4c-b367-04408d661523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:13.790259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dbe5b-75bc-4c4c-b367-04408d661523 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.790566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dbe5b-75bc-4c4c-b367-04408d661523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.790867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dbe5b-75bc-4c4c-b367-04408d661523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.792043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dbe5b-75bc-4c4c-b367-04408d661523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:13.793785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dbe5b-75bc-4c4c-b367-04408d661523 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:13.819912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:13.830977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 923/1866] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:13.831170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e869c0-0dc7-4d59-8db3-2b528616f6ea admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:13.837297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:13.838055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.838343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.838624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] PATH_INFO: `/v3/projects/bcaf4d142ace4336894a100a0955712a/users/5277212f30eb4b8da78ff5840cfdaaf6/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.856604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcaf4d142ace4336894a100a0955712a, user_id=5277212f30eb4b8da78ff5840cfdaaf6, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:13.861565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:13.885879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 944/1867] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:13 2022] PUT /identity/v3/projects/bcaf4d142ace4336894a100a0955712a/users/5277212f30eb4b8da78ff5840cfdaaf6/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:13.891319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afdbef-d043-40e1-a685-ae2921815b51 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:13.891614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afdbef-d043-40e1-a685-ae2921815b51 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:13.891897 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afdbef-d043-40e1-a685-ae2921815b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:13.926007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a2afdbef-d043-40e1-a685-ae2921815b51 None None] MFA Rules not processed for user `5277212f30eb4b8da78ff5840cfdaaf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:13.927783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a2afdbef-d043-40e1-a685-ae2921815b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:14.056836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 924/1868] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:14.057666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:14.164605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1fb22-ada2-4916-b5ae-0cb525e58fc2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:14.169965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:14.170948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.171390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.171816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.172669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:14.174908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:14.227142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 945/1869] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:14 2022] POST /identity/v3/projects => generated 389 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:14.231091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afdbef-d043-40e1-a685-ae2921815b51 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:14.243003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afdbef-d043-40e1-a685-ae2921815b51 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:14.339261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:14.341617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.342033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.342430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.343625 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:14.345800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:14.386569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 925/1870] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:14 2022] POST /identity/v3/users => generated 301 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:14.388236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:14.401874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211d503-4b70-44c1-a455-bef4661ef49a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:14.415100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:14.415722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.416000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.416276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.417046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:14.418542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:14.424265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:14.430990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 946/1871] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:14.435576 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:14.462925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca1c43-5690-4f95-ad43-4de733894f0b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:14.479100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:14.479812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.480120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.480395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] PATH_INFO: `/v3/projects/ee9ed3ed0bec4cd492ad51ec453d9040/users/14e45de5b7fa496995bdbbce963ab626/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.514138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9ed3ed0bec4cd492ad51ec453d9040, user_id=14e45de5b7fa496995bdbbce963ab626, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:14.521479 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:14.543246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 926/1872] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:14 2022] PUT /identity/v3/projects/ee9ed3ed0bec4cd492ad51ec453d9040/users/14e45de5b7fa496995bdbbce963ab626/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:14.547240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:14.567418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347c2ed-44af-4937-aeee-ec90ad7de523 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:14.678932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97383077-43c6-4022-a8d7-e899e34239e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:14.679606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383077-43c6-4022-a8d7-e899e34239e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.679813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383077-43c6-4022-a8d7-e899e34239e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.680089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97383077-43c6-4022-a8d7-e899e34239e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.680805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97383077-43c6-4022-a8d7-e899e34239e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:14.682637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97383077-43c6-4022-a8d7-e899e34239e1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:14.705633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 947/1873] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:14.711078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:14.722359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ea4e1-9c29-43f5-89be-5d254731a18f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:14.728124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:14.728916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.729287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.729678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] PATH_INFO: `/v3/projects/ee9ed3ed0bec4cd492ad51ec453d9040/users/14e45de5b7fa496995bdbbce963ab626/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.731822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9ed3ed0bec4cd492ad51ec453d9040, user_id=14e45de5b7fa496995bdbbce963ab626, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:14.735199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:14.753839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 927/1874] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:14 2022] PUT /identity/v3/projects/ee9ed3ed0bec4cd492ad51ec453d9040/users/14e45de5b7fa496995bdbbce963ab626/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:14.762907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14287d00-49f9-41ef-9443-e18a0f0c6f62 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.763250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14287d00-49f9-41ef-9443-e18a0f0c6f62 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.763552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14287d00-49f9-41ef-9443-e18a0f0c6f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.796415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-14287d00-49f9-41ef-9443-e18a0f0c6f62 None None] MFA Rules not processed for user `14e45de5b7fa496995bdbbce963ab626`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:14.797870 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-14287d00-49f9-41ef-9443-e18a0f0c6f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:14.880829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 948/1875] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:14.893078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:14.895913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:14.933843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:14.964680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b972e65-eb6e-4690-8def-23d37fb87a3d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:14.977859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea435b5b-817f-42af-b343-f20a2efb8a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:14.977859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea435b5b-817f-42af-b343-f20a2efb8a9a service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:14.977859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea435b5b-817f-42af-b343-f20a2efb8a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:14.977859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea435b5b-817f-42af-b343-f20a2efb8a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:14.980292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea435b5b-817f-42af-b343-f20a2efb8a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:14.981830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea435b5b-817f-42af-b343-f20a2efb8a9a service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:14.991715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 928/1876] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:14 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:15.141158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14287d00-49f9-41ef-9443-e18a0f0c6f62 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:15.205383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14287d00-49f9-41ef-9443-e18a0f0c6f62 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:15.210543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:15.211177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:15.211455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:15.211724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] PATH_INFO: `/v3/users/5277212f30eb4b8da78ff5840cfdaaf6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:15.212844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5277212f30eb4b8da78ff5840cfdaaf6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:15.214834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:15.241788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:15.267703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5277212f30eb4b8da78ff5840cfdaaf6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:15.275372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5277212f30eb4b8da78ff5840cfdaaf6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:15.288727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5277212f30eb4b8da78ff5840cfdaaf6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:15.296828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5277212f30eb4b8da78ff5840cfdaaf6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:15.297872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5277212f30eb4b8da78ff5840cfdaaf6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:15.306736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 949/1877] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:15 2022] DELETE /identity/v3/users/5277212f30eb4b8da78ff5840cfdaaf6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:15.310969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:15.312337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:15.312818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:15.360661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:15.361975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:15.444124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 929/1878] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:15.453836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:15.455494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:15.528435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0ea6-dd3b-4790-a641-a56e5f13d02a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:15.532260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:15.532887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:15.533313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:15.533591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:15.545183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:15.546658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:15.558681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 950/1879] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:16.062920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:16.064466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:16.079214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64cea1-27ba-43dc-a939-7fbcbca76b61 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:16.087493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:16.105531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:16.106805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:16.107848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] PATH_INFO: `/v3/projects/bcaf4d142ace4336894a100a0955712a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:16.111690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcaf4d142ace4336894a100a0955712a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:16.114971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:16.236032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcaf4d142ace4336894a100a0955712a', 'request_id': 'req-83440fc5-6660-42dd-99dc-6522ca454980'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.252716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcaf4d142ace4336894a100a0955712a', 'request_id': 'req-83440fc5-6660-42dd-99dc-6522ca454980'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.262313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcaf4d142ace4336894a100a0955712a', 'request_id': 'req-83440fc5-6660-42dd-99dc-6522ca454980'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.281790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] The token cache is being invalidate because project bcaf4d142ace4336894a100a0955712a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:16.282742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.287279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.293596 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:16.294106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 930/1880] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:16 2022] DELETE /identity/v3/projects/bcaf4d142ace4336894a100a0955712a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:16.304935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594dff1-43e9-43d8-847d-e6a2a74f0420 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:16.372524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:16.373357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:16.373715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:16.374088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] PATH_INFO: `/v3/users/14e45de5b7fa496995bdbbce963ab626` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:16.375303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e45de5b7fa496995bdbbce963ab626)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:16.378083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:16.447445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:16.450240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e45de5b7fa496995bdbbce963ab626'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.460470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e45de5b7fa496995bdbbce963ab626'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.478461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e45de5b7fa496995bdbbce963ab626'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.489313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e45de5b7fa496995bdbbce963ab626'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.490063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e667dfaf-bf59-4cef-bfa6-78c5018856c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e45de5b7fa496995bdbbce963ab626'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.497824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 951/1881] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:16 2022] DELETE /identity/v3/users/14e45de5b7fa496995bdbbce963ab626 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:16.777043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:16.778567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:16.797844 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440fc5-6660-42dd-99dc-6522ca454980 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:16.846641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:16.857858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:16.858520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:16.858799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:16.859076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] PATH_INFO: `/v3/projects/ee9ed3ed0bec4cd492ad51ec453d9040` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:16.860223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9ed3ed0bec4cd492ad51ec453d9040)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:16.862181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:16.910898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9ed3ed0bec4cd492ad51ec453d9040', 'request_id': 'req-cddf13a3-b06d-4509-92a9-020584bf0a38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.917155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9ed3ed0bec4cd492ad51ec453d9040', 'request_id': 'req-cddf13a3-b06d-4509-92a9-020584bf0a38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.926525 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9ed3ed0bec4cd492ad51ec453d9040', 'request_id': 'req-cddf13a3-b06d-4509-92a9-020584bf0a38'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.940289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] The token cache is being invalidate because project ee9ed3ed0bec4cd492ad51ec453d9040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:16.940614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.941813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-cddf13a3-b06d-4509-92a9-020584bf0a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:16.943542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 931/1882] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:16 2022] DELETE /identity/v3/projects/ee9ed3ed0bec4cd492ad51ec453d9040 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:16.948438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aab053-cb3d-4ac9-8634-7e5a0e60965e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:16.948723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aab053-cb3d-4ac9-8634-7e5a0e60965e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:16.948997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aab053-cb3d-4ac9-8634-7e5a0e60965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:16.969244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e7aab053-cb3d-4ac9-8634-7e5a0e60965e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:16.970795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e7aab053-cb3d-4ac9-8634-7e5a0e60965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:17.063575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 952/1883] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:17.065531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939e10-9d35-45b4-8ef4-34799953c908 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.066510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939e10-9d35-45b4-8ef4-34799953c908 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.066890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f939e10-9d35-45b4-8ef4-34799953c908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.101164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9f939e10-9d35-45b4-8ef4-34799953c908 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:17.103960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9f939e10-9d35-45b4-8ef4-34799953c908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:17.114647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 932/1884] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:17.119315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aab053-cb3d-4ac9-8634-7e5a0e60965e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.130115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aab053-cb3d-4ac9-8634-7e5a0e60965e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.135595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.136264 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.136549 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.136851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.137475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:17.137825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.139446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.152848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 953/1885] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:17.157224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939e10-9d35-45b4-8ef4-34799953c908 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.167002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f939e10-9d35-45b4-8ef4-34799953c908 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.171926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.172532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.172815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.173093 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.173751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.176884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.203456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 933/1886] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:17 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:17.207806 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.217805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85676ee-c99f-4929-814f-9767a23bb7ea admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.274026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.274609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.274890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.275164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.275803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.277700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.305685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 954/1887] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:17 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:17.306948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.318206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be84c1-c7a0-4ebf-b3ee-e33c272fb45f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.324490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.325177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.325522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.325905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.326617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.328238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.353048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 934/1888] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:17.354609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.367702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e290dba-7b01-40c3-961a-cb5441306beb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.379699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5d787-77a2-49f6-bd0b-d2be8a2e1f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.380397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5d787-77a2-49f6-bd0b-d2be8a2e1f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.380677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5d787-77a2-49f6-bd0b-d2be8a2e1f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.380950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5d787-77a2-49f6-bd0b-d2be8a2e1f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.381771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5d787-77a2-49f6-bd0b-d2be8a2e1f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.383576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5d787-77a2-49f6-bd0b-d2be8a2e1f61 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.397704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 955/1889] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:17.401527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.414347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fb127-e149-417b-9de0-51bd522aa612 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.421174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.421996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.422352 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.422779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] PATH_INFO: `/v3/projects/3ae9a6ebc77942328751c4ef2828e22a/users/06e23bb8ae464dfc9215f36003c27181/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.445221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae9a6ebc77942328751c4ef2828e22a, user_id=06e23bb8ae464dfc9215f36003c27181, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.448336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.466266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 935/1890] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:17 2022] PUT /identity/v3/projects/3ae9a6ebc77942328751c4ef2828e22a/users/06e23bb8ae464dfc9215f36003c27181/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:17.478334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9aca07-401f-4c89-af48-0a8d6d7e2a19 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.478616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9aca07-401f-4c89-af48-0a8d6d7e2a19 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.478894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9aca07-401f-4c89-af48-0a8d6d7e2a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.524463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ba9aca07-401f-4c89-af48-0a8d6d7e2a19 None None] MFA Rules not processed for user `06e23bb8ae464dfc9215f36003c27181`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:17.525676 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ba9aca07-401f-4c89-af48-0a8d6d7e2a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:17.543639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:17.594802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 956/1891] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:17.599437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.662578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc11b4a-e880-4ef0-bba8-c04390a87d56 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.667363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.667960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.668242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.668515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.669141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.671062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.698708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 936/1892] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:17 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:17.702758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9aca07-401f-4c89-af48-0a8d6d7e2a19 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.713477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9aca07-401f-4c89-af48-0a8d6d7e2a19 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.768243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.768839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.769379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.769656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.770308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.772706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.793019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 957/1893] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:17 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:17.797076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.804204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:17.808617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba15fbe-8991-471f-a57a-a920a6c1e3ef admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.814800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.815434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.816526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.816984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.817809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.820498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.831075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 937/1894] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:17.835504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.845592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848c490-bbd9-48c0-a8de-5d17b6193705 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.851123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.851784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.852079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.852344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] PATH_INFO: `/v3/projects/5906cca8d65446cdb1566a7925b8e662/users/8ff6ebaa407c47c887824e3fcfbbe42c/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.868677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5906cca8d65446cdb1566a7925b8e662, user_id=8ff6ebaa407c47c887824e3fcfbbe42c, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.871403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.885288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 958/1895] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:17 2022] PUT /identity/v3/projects/5906cca8d65446cdb1566a7925b8e662/users/8ff6ebaa407c47c887824e3fcfbbe42c/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:17.889310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.899740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697d38e-1b39-49d1-9f1c-38454dd4e286 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.955373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2d1f5-2569-4477-927b-771a51362056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.956079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2d1f5-2569-4477-927b-771a51362056 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.956374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2d1f5-2569-4477-927b-771a51362056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.956651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2d1f5-2569-4477-927b-771a51362056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.957404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2d1f5-2569-4477-927b-771a51362056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.959048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2d1f5-2569-4477-927b-771a51362056 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:17.969293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 938/1896] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:17.974215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:17.984455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96947-90c4-4105-863f-942daae275fc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:17.989791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:17.990440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:17.990719 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:17.990993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] PATH_INFO: `/v3/projects/5906cca8d65446cdb1566a7925b8e662/users/8ff6ebaa407c47c887824e3fcfbbe42c/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:17.992908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5906cca8d65446cdb1566a7925b8e662, user_id=8ff6ebaa407c47c887824e3fcfbbe42c, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:17.995570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:18.006944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 959/1897] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:17 2022] PUT /identity/v3/projects/5906cca8d65446cdb1566a7925b8e662/users/8ff6ebaa407c47c887824e3fcfbbe42c/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:18.018632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219b33a-b97c-42a0-bc53-082394178f7b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:18.018997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219b33a-b97c-42a0-bc53-082394178f7b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:18.019279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219b33a-b97c-42a0-bc53-082394178f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:18.047385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e219b33a-b97c-42a0-bc53-082394178f7b None None] MFA Rules not processed for user `8ff6ebaa407c47c887824e3fcfbbe42c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:18.048592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e219b33a-b97c-42a0-bc53-082394178f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:18.120862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 939/1898] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:18.132005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:18.133786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:18.199143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf6615-c5ca-4efe-91d1-677dca49f448 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:18.203052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:18.203557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:18.203963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:18.204245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:18.214936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:18.216407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:18.228276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 960/1899] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:18.343562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219b33a-b97c-42a0-bc53-082394178f7b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:18.354319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219b33a-b97c-42a0-bc53-082394178f7b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:18.358108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4eafd-132d-48aa-90da-703fd98a3cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:18.358723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4eafd-132d-48aa-90da-703fd98a3cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:18.359410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4eafd-132d-48aa-90da-703fd98a3cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:18.359410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4eafd-132d-48aa-90da-703fd98a3cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:18.435392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4eafd-132d-48aa-90da-703fd98a3cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:18.437151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4eafd-132d-48aa-90da-703fd98a3cf6 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:18.449282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 940/1900] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:23.257901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:23.272295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:23.323777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e85ed-e18a-468e-a184-0a8a14a5b866 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:23.328649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:23.329286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:23.329568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:23.329871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] PATH_INFO: `/v3/users/06e23bb8ae464dfc9215f36003c27181` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:23.331015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e23bb8ae464dfc9215f36003c27181)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:23.333297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:23.384350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e23bb8ae464dfc9215f36003c27181'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.391268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e23bb8ae464dfc9215f36003c27181'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.404237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e23bb8ae464dfc9215f36003c27181'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.413334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e23bb8ae464dfc9215f36003c27181'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.413982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e23bb8ae464dfc9215f36003c27181'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.420739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 961/1901] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:23 2022] DELETE /identity/v3/users/06e23bb8ae464dfc9215f36003c27181 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:23.425485 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:23.426006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:23.426362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:23.449929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:23.451132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:23.460308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:23.519693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 941/1902] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:23.530448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:23.531917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:23.590519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c370a19-b0ad-4285-b7c8-ae389fed6c47 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:23.594372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:23.594959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:23.595235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:23.595503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:23.606079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:23.607490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:23.616872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 962/1903] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:23.886398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:23.887836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:23.901918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257bbef4-d80c-403c-bda0-79e408d1cae5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:23.906807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:23.907440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:23.907730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:23.908733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] PATH_INFO: `/v3/projects/3ae9a6ebc77942328751c4ef2828e22a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:23.910453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae9a6ebc77942328751c4ef2828e22a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:23.912692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:23.962724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae9a6ebc77942328751c4ef2828e22a', 'request_id': 'req-d91e16c1-99b4-464d-91cf-cb18e0f121f8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.968869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae9a6ebc77942328751c4ef2828e22a', 'request_id': 'req-d91e16c1-99b4-464d-91cf-cb18e0f121f8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.977195 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae9a6ebc77942328751c4ef2828e22a', 'request_id': 'req-d91e16c1-99b4-464d-91cf-cb18e0f121f8'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.988680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] The token cache is being invalidate because project 3ae9a6ebc77942328751c4ef2828e22a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:23.988977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.989644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:23.991233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 942/1904] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:23 2022] DELETE /identity/v3/projects/3ae9a6ebc77942328751c4ef2828e22a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:23.995301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:24.005894 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99589a76-57b6-46da-a975-36a10a2a940f service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:24.039381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:24.067823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:24.068611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.068988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.069178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] PATH_INFO: `/v3/users/8ff6ebaa407c47c887824e3fcfbbe42c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.070401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff6ebaa407c47c887824e3fcfbbe42c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:24.072355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:24.125053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff6ebaa407c47c887824e3fcfbbe42c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.136224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff6ebaa407c47c887824e3fcfbbe42c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.150553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff6ebaa407c47c887824e3fcfbbe42c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.158601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff6ebaa407c47c887824e3fcfbbe42c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.159418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6da37b0f-5c8a-40c5-9693-cf375c4f1730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff6ebaa407c47c887824e3fcfbbe42c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.166016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 963/1905] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:23 2022] DELETE /identity/v3/users/8ff6ebaa407c47c887824e3fcfbbe42c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:24.492239 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:24.493949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:24.508901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e16c1-99b4-464d-91cf-cb18e0f121f8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:24.577284 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:24.577996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.578302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.578581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] PATH_INFO: `/v3/projects/5906cca8d65446cdb1566a7925b8e662` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.579727 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5906cca8d65446cdb1566a7925b8e662)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:24.582537 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:24.625915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:24.634758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5906cca8d65446cdb1566a7925b8e662', 'request_id': 'req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.640587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5906cca8d65446cdb1566a7925b8e662', 'request_id': 'req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.648377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5906cca8d65446cdb1566a7925b8e662', 'request_id': 'req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.660753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] The token cache is being invalidate because project 5906cca8d65446cdb1566a7925b8e662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:24.661065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.661720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-846eeb2b-0dc4-476b-aeb2-8b6e01e81e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:24.663562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 943/1906] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:24 2022] DELETE /identity/v3/projects/5906cca8d65446cdb1566a7925b8e662 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:24.668096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb2630-420c-4c68-a82a-52f1fc757cb4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.668561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb2630-420c-4c68-a82a-52f1fc757cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.668650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb2630-420c-4c68-a82a-52f1fc757cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.689099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6ebb2630-420c-4c68-a82a-52f1fc757cb4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:24.691003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6ebb2630-420c-4c68-a82a-52f1fc757cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:24.756389 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 964/1907] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:24.764117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293a50d-c1bd-4066-8fc4-5b7c3c658057 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.764403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293a50d-c1bd-4066-8fc4-5b7c3c658057 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.764681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293a50d-c1bd-4066-8fc4-5b7c3c658057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.784907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-c293a50d-c1bd-4066-8fc4-5b7c3c658057 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:24.786138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c293a50d-c1bd-4066-8fc4-5b7c3c658057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:24.795677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 944/1908] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:24.800490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb2630-420c-4c68-a82a-52f1fc757cb4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:24.810562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb2630-420c-4c68-a82a-52f1fc757cb4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:24.815338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:24.816171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.816625 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.816906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.817565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:24.817903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:24.819494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:24.831247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 965/1909] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:24.835768 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293a50d-c1bd-4066-8fc4-5b7c3c658057 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:24.844947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293a50d-c1bd-4066-8fc4-5b7c3c658057 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:24.850252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:24.850841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.851118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.851390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.852035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:24.853933 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:24.880752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 945/1910] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:24 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:24.884854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:24.895895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7a9a7-91b1-4633-a663-1f9482412169 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:24.934783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:24.953109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:24.953708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.954021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.954293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.954922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:24.957451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:24.978416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 966/1911] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:24 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:24.982012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:24.992811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed63de-1a10-4afd-9b01-f276e57abc96 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:24.997431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:24.998057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:24.998337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:24.998608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:24.999241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.000810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.011416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 946/1912] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:25.015028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:25.025397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee7493-c7af-41a8-ac61-c0689056df45 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:25.030209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12e362-91a7-425b-9172-fd9b67dc8946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:25.030803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12e362-91a7-425b-9172-fd9b67dc8946 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.031084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12e362-91a7-425b-9172-fd9b67dc8946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.031360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12e362-91a7-425b-9172-fd9b67dc8946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.031996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12e362-91a7-425b-9172-fd9b67dc8946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.033569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12e362-91a7-425b-9172-fd9b67dc8946 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.044294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 967/1913] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:25.048253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:25.058522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d88430-5d8d-43f7-a456-f7768ff9daf3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:25.063123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:25.063759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.064036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.064310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] PATH_INFO: `/v3/projects/1ddfa50f8cad427e90b7244428bca6e3/users/51c72e123c9b4891a3eca9aedeaa36ed/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.081261 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddfa50f8cad427e90b7244428bca6e3, user_id=51c72e123c9b4891a3eca9aedeaa36ed, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.084515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.097289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 947/1914] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:25 2022] PUT /identity/v3/projects/1ddfa50f8cad427e90b7244428bca6e3/users/51c72e123c9b4891a3eca9aedeaa36ed/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:25.107599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.107882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.108152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.137347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] MFA Rules not processed for user `51c72e123c9b4891a3eca9aedeaa36ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:25.138517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:25.203557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 968/1915] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:25.214570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:25.216723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:25.277036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4e7ad-f171-41c9-ad81-cd18050f2757 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:25.283807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:25.284734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.285141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.285423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.289851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:25.291335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:25.297220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.298695 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.308619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 948/1916] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:25.331376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:25.357436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bdab6-baf4-44a6-b4d6-e8b6ac756894 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:25.362252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:25.363489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.363773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.364041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] PATH_INFO: `/v3/users/4866f815cf7044d498685a12c5d31df0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.365181 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4866f815cf7044d498685a12c5d31df0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.367210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.396558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1e44-cc0c-42e1-ad31-5bd5b5eb9bde service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:25.401452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:25.402118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.402430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.402704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] PATH_INFO: `/v3/users/51c72e123c9b4891a3eca9aedeaa36ed` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.403880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c72e123c9b4891a3eca9aedeaa36ed)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.405842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.412236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:25.414602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4866f815cf7044d498685a12c5d31df0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.419229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:25.428149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4866f815cf7044d498685a12c5d31df0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.443497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4866f815cf7044d498685a12c5d31df0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.451116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4866f815cf7044d498685a12c5d31df0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.451889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6f293179-279e-4045-88be-aa430f44f0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4866f815cf7044d498685a12c5d31df0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.458144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 969/1917] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:25 2022] DELETE /identity/v3/users/4866f815cf7044d498685a12c5d31df0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:25.461157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c72e123c9b4891a3eca9aedeaa36ed'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.462469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.462760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.463036 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.468338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c72e123c9b4891a3eca9aedeaa36ed'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.481893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c72e123c9b4891a3eca9aedeaa36ed'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.482725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c72e123c9b4891a3eca9aedeaa36ed'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.483430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-91849b01-0cfb-42c0-8340-7a0a3efb0348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c72e123c9b4891a3eca9aedeaa36ed'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:25.483544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:25.484924 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:25.499939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 949/1918] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:25 2022] DELETE /identity/v3/users/51c72e123c9b4891a3eca9aedeaa36ed => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:25.505412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.505941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.506416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.553968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:25.555700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:25.563030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 970/1919] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:25.585210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:25.587313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:25.631156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 950/1920] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:25.641364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:25.643321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:25.677982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19781a-9db5-415c-ba03-0a909d07ec76 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:25.710384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797be522-e6df-40a2-bbbd-35648e14e823 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:25.714164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:25.714780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.715062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.715338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.725573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.727196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.739779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 951/1921] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:25.740008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb8b4-0dc9-4a7e-8145-7013d36e37bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:25.740626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb8b4-0dc9-4a7e-8145-7013d36e37bf service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:25.740906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb8b4-0dc9-4a7e-8145-7013d36e37bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:25.741179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb8b4-0dc9-4a7e-8145-7013d36e37bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:25.751104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb8b4-0dc9-4a7e-8145-7013d36e37bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:25.752682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb8b4-0dc9-4a7e-8145-7013d36e37bf service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:25.764298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 971/1922] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:26.187495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.189320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.205582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967960b-73d3-4702-8e54-960b9c0fa08d service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.211159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.212256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.212703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.213139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] PATH_INFO: `/v3/projects/b75869fe5009408383d57e72c4c5b953` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.214712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75869fe5009408383d57e72c4c5b953)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.216851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.248509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:26.290689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75869fe5009408383d57e72c4c5b953', 'request_id': 'req-701ea938-85bd-413c-845c-6edb05c79833'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.298025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75869fe5009408383d57e72c4c5b953', 'request_id': 'req-701ea938-85bd-413c-845c-6edb05c79833'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.306987 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75869fe5009408383d57e72c4c5b953', 'request_id': 'req-701ea938-85bd-413c-845c-6edb05c79833'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.320111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] The token cache is being invalidate because project b75869fe5009408383d57e72c4c5b953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:26.320469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.321194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.322840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 952/1923] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:26 2022] DELETE /identity/v3/projects/b75869fe5009408383d57e72c4c5b953 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:26.328514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20b29-3027-4905-a1f7-bf57c62bc182 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.328989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20b29-3027-4905-a1f7-bf57c62bc182 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.329323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d20b29-3027-4905-a1f7-bf57c62bc182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.350758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-93d20b29-3027-4905-a1f7-bf57c62bc182 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:26.351981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-93d20b29-3027-4905-a1f7-bf57c62bc182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.376886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:26.387038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.389470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.419973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 972/1924] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:26.427891 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ff93e-acf5-4006-be35-38b0d5ca5633 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.428180 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ff93e-acf5-4006-be35-38b0d5ca5633 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.428737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ff93e-acf5-4006-be35-38b0d5ca5633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.458967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-013ff93e-acf5-4006-be35-38b0d5ca5633 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:26.460558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-013ff93e-acf5-4006-be35-38b0d5ca5633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.470928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 973/1925] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:26.475520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ff93e-acf5-4006-be35-38b0d5ca5633 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.484751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ff93e-acf5-4006-be35-38b0d5ca5633 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.490240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.490935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.491282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.491638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.492340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:26.492643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.494517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.496739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ea938-85bd-413c-845c-6edb05c79833 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.502186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.502932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.503956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.504372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] PATH_INFO: `/v3/projects/1ddfa50f8cad427e90b7244428bca6e3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.505567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ddfa50f8cad427e90b7244428bca6e3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.507489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.513780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 974/1926] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:26.515893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.530795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4040-c598-45f8-a39c-2c9cb674b01f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.535883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.536651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.537030 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.538127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.538127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.540478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.567369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ddfa50f8cad427e90b7244428bca6e3', 'request_id': 'req-79fedbe2-ded0-42ef-9a45-3108f875ccea'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.570706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 975/1927] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:26.574699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.576746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ddfa50f8cad427e90b7244428bca6e3', 'request_id': 'req-79fedbe2-ded0-42ef-9a45-3108f875ccea'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.588045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ddfa50f8cad427e90b7244428bca6e3', 'request_id': 'req-79fedbe2-ded0-42ef-9a45-3108f875ccea'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.610411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] The token cache is being invalidate because project 1ddfa50f8cad427e90b7244428bca6e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:26.610707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.611495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-79fedbe2-ded0-42ef-9a45-3108f875ccea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:26.614985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 953/1928] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:26 2022] DELETE /identity/v3/projects/1ddfa50f8cad427e90b7244428bca6e3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:26.620927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc1620-c396-4df0-a52a-d6215ab14b32 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.621216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc1620-c396-4df0-a52a-d6215ab14b32 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.621495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc1620-c396-4df0-a52a-d6215ab14b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.649687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96b39c-97ac-4f8f-95fe-d00d959df180 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.651167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8abc1620-c396-4df0-a52a-d6215ab14b32 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:26.652914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8abc1620-c396-4df0-a52a-d6215ab14b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.655413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.655926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.656700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.656700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.657343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.660419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.669206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 954/1929] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:26.675346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d13210-dadf-447b-98fb-bd955ff71c6b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.675642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d13210-dadf-447b-98fb-bd955ff71c6b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.675921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d13210-dadf-447b-98fb-bd955ff71c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.702367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-07d13210-dadf-447b-98fb-bd955ff71c6b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:26.703737 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-07d13210-dadf-447b-98fb-bd955ff71c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.713928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 976/1930] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:26.719922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.720220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 955/1931] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:26.725218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.727014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d13210-dadf-447b-98fb-bd955ff71c6b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.740476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d13210-dadf-447b-98fb-bd955ff71c6b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.745012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba2bb9-dedc-49bf-80ae-88de4aa3fe96 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.746258 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.746959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.747294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.747653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.748386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:26.748771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.750356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.750491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.750973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.751259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.751533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.752183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.754178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.772041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 956/1932] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:26.772041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.772740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 977/1933] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:26.773992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.785834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffeabc-f07b-4fc7-b82c-ca34eec6bad8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.789375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af9c63-6df9-4b55-a97a-1bec1d5bbf22 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.791053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.792019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.792288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.792568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.793222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.795097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.797174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.798134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.798564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.798989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.800996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.803152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.809017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 978/1934] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:26.813060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.824896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:26.829092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecb864-8f78-4962-97a9-ea3af7477eb6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.835087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582dfe14-5968-494f-94fd-184b0a1ed0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.835762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dfe14-5968-494f-94fd-184b0a1ed0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.836107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dfe14-5968-494f-94fd-184b0a1ed0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.836456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dfe14-5968-494f-94fd-184b0a1ed0aa admin admin] PATH_INFO: `/v3/projects/97c52c65a3a347d0b9a0ca3852efaa46/users/b08dad1bc98f4eb28b5e4d055707620d/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.844722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 957/1935] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:26.845743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:26.858922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582dfe14-5968-494f-94fd-184b0a1ed0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c52c65a3a347d0b9a0ca3852efaa46, user_id=b08dad1bc98f4eb28b5e4d055707620d, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:26.864556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582dfe14-5968-494f-94fd-184b0a1ed0aa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:26.885463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 979/1936] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:26 2022] PUT /identity/v3/projects/97c52c65a3a347d0b9a0ca3852efaa46/users/b08dad1bc98f4eb28b5e4d055707620d/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:26.897062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72ed4-e76f-462d-9660-740db691b3d4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.897426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72ed4-e76f-462d-9660-740db691b3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.897768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72ed4-e76f-462d-9660-740db691b3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.922852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8c60-c12b-4bd2-929b-d44cea76251a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:26.939954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-46a72ed4-e76f-462d-9660-740db691b3d4 None None] MFA Rules not processed for user `b08dad1bc98f4eb28b5e4d055707620d`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:26.942582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-46a72ed4-e76f-462d-9660-740db691b3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:26.972787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:26.995992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:26.996607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:26.996884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:26.997222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:26.998059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.000903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.040393 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 958/1937] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:27.044713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.058877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8ccb2-8e1d-4f55-9ccb-f626dc901c89 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.065246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.067223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.067805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.068227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.069406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.071347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.077240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 980/1938] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:27.078387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a72ed4-e76f-462d-9660-740db691b3d4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.086749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 959/1939] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:27.088009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.103263 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5464b-5738-41ad-bbf0-a2b9462d0b68 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.103701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a72ed4-e76f-462d-9660-740db691b3d4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.108271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.108878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.109156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.109428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.110110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.111718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.114301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.115428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.115776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.116117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.116848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.119421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.126955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 960/1940] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:27.131806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.143814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520e57b-d809-43a4-81d7-b8fca9fe092d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.150578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c2ca3-8163-4875-a846-dfa54c3cbb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.151282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2ca3-8163-4875-a846-dfa54c3cbb16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.151549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2ca3-8163-4875-a846-dfa54c3cbb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.152346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2ca3-8163-4875-a846-dfa54c3cbb16 admin admin] PATH_INFO: `/v3/projects/36e5d228c4714f4a9baba0faa4c46d88/users/45d9b6bd4eea408db0f0b9df7fd7e89d/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.173641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 981/1941] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:27.175195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.177706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c2ca3-8163-4875-a846-dfa54c3cbb16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e5d228c4714f4a9baba0faa4c46d88, user_id=45d9b6bd4eea408db0f0b9df7fd7e89d, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.183650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c2ca3-8163-4875-a846-dfa54c3cbb16 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.204885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 961/1942] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:27 2022] PUT /identity/v3/projects/36e5d228c4714f4a9baba0faa4c46d88/users/45d9b6bd4eea408db0f0b9df7fd7e89d/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:27.222589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c206a12-b3df-4af8-bf97-47e27f53e164 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.224437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c206a12-b3df-4af8-bf97-47e27f53e164 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.225014 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c206a12-b3df-4af8-bf97-47e27f53e164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.267836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3c206a12-b3df-4af8-bf97-47e27f53e164 None None] MFA Rules not processed for user `45d9b6bd4eea408db0f0b9df7fd7e89d`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:27.270048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c206a12-b3df-4af8-bf97-47e27f53e164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:27.287346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ee4a0-c603-4fce-b378-9e364dc4a441 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.311765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:27.350464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.351376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.351816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.352090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.352725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.354563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 962/1943] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:27.354684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.358807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c206a12-b3df-4af8-bf97-47e27f53e164 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.371848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c206a12-b3df-4af8-bf97-47e27f53e164 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.376844 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.377581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.378246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.381586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 982/1944] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:27.381715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.381715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.381715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.382745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.392895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237175-1885-44be-9203-86a8ac0c9905 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.399566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.400261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.400722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.401090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.401841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.403514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.416177 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 963/1945] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:27.416367 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 983/1946] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:27.418651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.418792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.430267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f7f76-a2a6-4b64-9db0-bf30439a2218 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.455731 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:27.485021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fd765-87f2-4bf5-82c3-90a78726454b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.489983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.490590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.490873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.491148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.491789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.493698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.495887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.496490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.496769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.497042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.497685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.499288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.509850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 984/1947] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:27.514024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.522663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 964/1948] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:27.525015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfd67a-5ae2-4e5f-b5f1-ddaa4ede2a1d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.530158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78ba95-792c-4713-85b3-ef3ae3394757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.530869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.530978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78ba95-792c-4713-85b3-ef3ae3394757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.531276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78ba95-792c-4713-85b3-ef3ae3394757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.531720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78ba95-792c-4713-85b3-ef3ae3394757 admin admin] PATH_INFO: `/v3/projects/cf5cf890a24e41aba06ab35ca359edc7/users/8608395ca01d464f9cc29dfff4c14b74/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.542564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4c526-ac7f-4566-bc43-236aa960bebb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.548327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.548465 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78ba95-792c-4713-85b3-ef3ae3394757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5cf890a24e41aba06ab35ca359edc7, user_id=8608395ca01d464f9cc29dfff4c14b74, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.548938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.549236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.549494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.551390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.551996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78ba95-792c-4713-85b3-ef3ae3394757 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.553002 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.563461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 965/1949] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:27.566822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 985/1950] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:27 2022] PUT /identity/v3/projects/cf5cf890a24e41aba06ab35ca359edc7/users/8608395ca01d464f9cc29dfff4c14b74/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:27.567480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.576998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70a1d2-53b9-4f0e-8266-07b581da4034 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.577289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70a1d2-53b9-4f0e-8266-07b581da4034 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.577564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70a1d2-53b9-4f0e-8266-07b581da4034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.579004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba20cb3-34cb-41ca-8d98-24426f94aaf3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.605984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3c70a1d2-53b9-4f0e-8266-07b581da4034 None None] MFA Rules not processed for user `8608395ca01d464f9cc29dfff4c14b74`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:27.607204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3c70a1d2-53b9-4f0e-8266-07b581da4034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:27.636758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.637612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.637929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.638201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.638835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.640449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.650628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 966/1951] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:27.654678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.665008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cbfe5-4497-4b0c-9c97-8aedfe2c87d0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.669757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bfffe-81e0-4555-89d9-a2fd0a632ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.670391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bfffe-81e0-4555-89d9-a2fd0a632ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.670696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bfffe-81e0-4555-89d9-a2fd0a632ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.670971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bfffe-81e0-4555-89d9-a2fd0a632ec2 admin admin] PATH_INFO: `/v3/projects/55a983ac7f93459f879e02aaec6666b0/users/b041ecce24794c019742100f023c90bb/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.677171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 986/1952] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:27.681523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70a1d2-53b9-4f0e-8266-07b581da4034 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.686135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bfffe-81e0-4555-89d9-a2fd0a632ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a983ac7f93459f879e02aaec6666b0, user_id=b041ecce24794c019742100f023c90bb, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.688761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bfffe-81e0-4555-89d9-a2fd0a632ec2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.692392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70a1d2-53b9-4f0e-8266-07b581da4034 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.697740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.698356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.698632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.698905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.699599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.701530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.702089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 967/1953] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:27 2022] PUT /identity/v3/projects/55a983ac7f93459f879e02aaec6666b0/users/b041ecce24794c019742100f023c90bb/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:27.714077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28075d84-2e4c-466d-a810-3b298de4c0c5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.714303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28075d84-2e4c-466d-a810-3b298de4c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.714581 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28075d84-2e4c-466d-a810-3b298de4c0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.721382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:27.739314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 987/1954] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:27.740302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.748085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-28075d84-2e4c-466d-a810-3b298de4c0c5 None None] MFA Rules not processed for user `b041ecce24794c019742100f023c90bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:27.749275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-28075d84-2e4c-466d-a810-3b298de4c0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:27.803233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d816180-bb0a-4839-a4c5-50f52646b72a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.807975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.808586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.808865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.809141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.809798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.811736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.816271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 968/1955] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:27.820674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28075d84-2e4c-466d-a810-3b298de4c0c5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.822801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:27.836051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28075d84-2e4c-466d-a810-3b298de4c0c5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.840846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.841447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.841753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.842027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.842658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.844541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.850552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 988/1956] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:27.854173 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.864673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe9015-aca0-4392-a874-4b83b73244a7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.869821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.870437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.870720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.870988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.871632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.873213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.879574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 969/1957] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:27.880607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.886733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 989/1958] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:27.887728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.897647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409bb57-32c8-41fc-b112-aa097bacdcb2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.944071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062ba62-b896-478e-b4e4-6a23183dc899 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.948659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.949256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.949536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.949832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.950501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.952428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.954663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:27.955315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:27.955594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:27.955867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] PATH_INFO: `/v3/projects/8fb09e8fce2f4826b6b10ecf90808626/users/60d7e02a81ec451584e5cf1932d4a4dd/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:27.970416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb09e8fce2f4826b6b10ecf90808626, user_id=60d7e02a81ec451584e5cf1932d4a4dd, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:27.973706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:27.974568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 970/1959] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:27 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:27.978495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:27.988928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff8003-ce9d-46e7-a477-7ef1ae3d7909 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:27.989659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 990/1960] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:27 2022] PUT /identity/v3/projects/8fb09e8fce2f4826b6b10ecf90808626/users/60d7e02a81ec451584e5cf1932d4a4dd/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:27.993445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:28.050845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.051549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.051908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.052248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.052989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.054681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.056756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:28.060223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802b81-d3ce-4405-9805-57152eb85517 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:28.065198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.067105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.067393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.067667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.068313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.071904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 971/1961] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:28.072027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.072746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:28.084436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 991/1962] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:28.085439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:28.088984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf6a7-d103-4b2d-a391-b9a26e9dd8ad admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:28.095069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25f23c-89ae-4a5e-b6bc-86ff97fb9d38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:28.095235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.095753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.096040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.096309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] PATH_INFO: `/v3/projects/2a67c5e89219434689d202cf30feeeb7/users/2df89e85651a4801871a9edc96edb8af/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.102618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d27a7f-62d9-4faa-a508-c9f477136294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.103291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27a7f-62d9-4faa-a508-c9f477136294 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.103567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27a7f-62d9-4faa-a508-c9f477136294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.103838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d27a7f-62d9-4faa-a508-c9f477136294 admin admin] PATH_INFO: `/v3/projects/8fb09e8fce2f4826b6b10ecf90808626/users/60d7e02a81ec451584e5cf1932d4a4dd/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.105878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d27a7f-62d9-4faa-a508-c9f477136294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb09e8fce2f4826b6b10ecf90808626, user_id=60d7e02a81ec451584e5cf1932d4a4dd, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.108536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d27a7f-62d9-4faa-a508-c9f477136294 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.114555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a67c5e89219434689d202cf30feeeb7, user_id=2df89e85651a4801871a9edc96edb8af, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.117873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.120012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 992/1963] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:28 2022] PUT /identity/v3/projects/8fb09e8fce2f4826b6b10ecf90808626/users/60d7e02a81ec451584e5cf1932d4a4dd/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:28.131302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdc4e0-24d8-4168-a0f1-2ef26f025d9a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.131600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdc4e0-24d8-4168-a0f1-2ef26f025d9a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.131876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdc4e0-24d8-4168-a0f1-2ef26f025d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.142250 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 972/1964] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:28 2022] PUT /identity/v3/projects/2a67c5e89219434689d202cf30feeeb7/users/2df89e85651a4801871a9edc96edb8af/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:28.147513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:28.171135 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a7bdc4e0-24d8-4168-a0f1-2ef26f025d9a None None] MFA Rules not processed for user `60d7e02a81ec451584e5cf1932d4a4dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:28.173295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a7bdc4e0-24d8-4168-a0f1-2ef26f025d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:28.226557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:28.242022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de4883-81d3-4084-a429-c4c8ed8f7f91 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:28.248281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.248975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.249322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.249660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.250606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.252460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.265329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 993/1965] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:28.274876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 973/1966] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:28.279928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:28.281816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:28.282092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdc4e0-24d8-4168-a0f1-2ef26f025d9a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:28.301644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdc4e0-24d8-4168-a0f1-2ef26f025d9a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:28.308366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3f06b-72d1-447f-9b50-2b3f9e5c5341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.309043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f06b-72d1-447f-9b50-2b3f9e5c5341 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.309325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f06b-72d1-447f-9b50-2b3f9e5c5341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.309599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f06b-72d1-447f-9b50-2b3f9e5c5341 admin admin] PATH_INFO: `/v3/projects/2a67c5e89219434689d202cf30feeeb7/users/2df89e85651a4801871a9edc96edb8af/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.311854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3f06b-72d1-447f-9b50-2b3f9e5c5341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a67c5e89219434689d202cf30feeeb7, user_id=2df89e85651a4801871a9edc96edb8af, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.314608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3f06b-72d1-447f-9b50-2b3f9e5c5341 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.329860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 994/1967] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:28 2022] PUT /identity/v3/projects/2a67c5e89219434689d202cf30feeeb7/users/2df89e85651a4801871a9edc96edb8af/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:28.338702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.338988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.339265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.372029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] MFA Rules not processed for user `2df89e85651a4801871a9edc96edb8af`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:28.372597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381374e2-08fc-44da-bebc-0ff0dd813850 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:28.373341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:28.441542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.442355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.442633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.442904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.457464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 995/1968] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:28.468901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:28.470343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:28.483641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd3213-f701-44ce-ba45-36b5b20c4d9e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:28.487539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:28.488322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:28.488752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:28.489204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:28.489920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:28.512868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.514331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.525495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 974/1969] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:13:28.560816 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:28.562689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:28.572999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 996/1970] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:13:30.101896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:30.110557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8e7d1-60c5-49a7-987e-5a311d95f01d service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:30.115563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923261c-229c-4d69-b819-fbc0b7fe04db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:30.116528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923261c-229c-4d69-b819-fbc0b7fe04db service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:30.116873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923261c-229c-4d69-b819-fbc0b7fe04db service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:30.117369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923261c-229c-4d69-b819-fbc0b7fe04db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:30.118839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:30.129100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56126fff-d1ef-4c8e-9152-ee35ff6227bc service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:30.132978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:30.133662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:30.134122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:30.134391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:30.182105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:30.183461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923261c-229c-4d69-b819-fbc0b7fe04db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:30.185063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923261c-229c-4d69-b819-fbc0b7fe04db service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:30.195516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 975/1971] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:30.201943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:30.203692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:30.217527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 997/1972] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:30.631724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dffa7-d149-4535-8138-075120633223 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:30.632164 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dffa7-d149-4535-8138-075120633223 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:30.632697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dffa7-d149-4535-8138-075120633223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:30.657790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-fc7dffa7-d149-4535-8138-075120633223 None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:30.659310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fc7dffa7-d149-4535-8138-075120633223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:30.736111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 976/1973] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:30.789158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:30.790832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:30.899208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968852a-37ba-4709-a3e9-8128711f348a service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:30.905021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:30.905828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:30.906190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:30.906661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:30.917605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:30.920445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:30.934114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 998/1974] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:13:31.132618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dffa7-d149-4535-8138-075120633223 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:31.151822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dffa7-d149-4535-8138-075120633223 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:31.156667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:31.164234 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:31.166235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:31.168763 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:31.170041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:31.170716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccfc1d-456b-4b1d-9031-0df1cd40580a service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:31.183111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfb702-b8a6-4e33-b185-329e83d8ea2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:31.184544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb702-b8a6-4e33-b185-329e83d8ea2b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:31.185057 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb702-b8a6-4e33-b185-329e83d8ea2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:31.186095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb702-b8a6-4e33-b185-329e83d8ea2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:31.201760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfb702-b8a6-4e33-b185-329e83d8ea2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:31.204301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfb702-b8a6-4e33-b185-329e83d8ea2b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:31.221265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 999/1975] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:31.284887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:31.287375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:31.298583 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 977/1976] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:31.703701 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede7246-6c0b-49c2-b6dd-4bd750ce5994 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:31.703992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede7246-6c0b-49c2-b6dd-4bd750ce5994 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:31.704271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede7246-6c0b-49c2-b6dd-4bd750ce5994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:31.725593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fede7246-6c0b-49c2-b6dd-4bd750ce5994 None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:31.726837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fede7246-6c0b-49c2-b6dd-4bd750ce5994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:31.734415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1000/1977] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 13 14:13:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:31.747716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:31.759276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7204d16-1713-45ca-900a-284455696ceb service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:31.764196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:31.764793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:31.765068 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:31.765339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:31.773638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:31.775232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:31.783142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 978/1978] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:13:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:13:32.026121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af43ec-791c-454c-baee-151e39a08f8f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:32.026435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af43ec-791c-454c-baee-151e39a08f8f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:32.026735 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af43ec-791c-454c-baee-151e39a08f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:32.047771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-86af43ec-791c-454c-baee-151e39a08f8f None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:32.049081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-86af43ec-791c-454c-baee-151e39a08f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:32.057049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1001/1979] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:13:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:32.069505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:32.075135 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:32.081811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf8e73-d61f-4daa-b7a2-f39cd532c86c service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:32.087367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:32.088063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:32.088413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:32.088753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:32.097710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:32.099287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:32.107566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 979/1980] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:13:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:13:32.515533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:32.516011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:32.516377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:32.544107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:32.546069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:32.556490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1002/1981] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 13 14:13:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:32.574482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:32.589999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e5b22-885c-42b2-a3ce-390088f34d71 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:32.596744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a98a1-8b6e-4eda-8be0-da480e6c5df9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:32.597498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a98a1-8b6e-4eda-8be0-da480e6c5df9 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:32.598165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a98a1-8b6e-4eda-8be0-da480e6c5df9 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:32.598453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a98a1-8b6e-4eda-8be0-da480e6c5df9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:32.608396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a98a1-8b6e-4eda-8be0-da480e6c5df9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:32.610313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a98a1-8b6e-4eda-8be0-da480e6c5df9 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:32.620756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 980/1982] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:13:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:13:34.906032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:34.907600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:34.996977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f20a1-0b5f-4b95-a9c5-4c2b19777b3a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:35.003068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:35.003954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.004282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.004519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] PATH_INFO: `/v3/users/606e27c7d61546e1b546bab7109157fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.005776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] RBAC: Authorizing `identity:delete_user(user_id=606e27c7d61546e1b546bab7109157fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:35.007742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:35.057799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:35.063994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '606e27c7d61546e1b546bab7109157fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.072511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '606e27c7d61546e1b546bab7109157fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.087318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '606e27c7d61546e1b546bab7109157fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.096323 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '606e27c7d61546e1b546bab7109157fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.097232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '606e27c7d61546e1b546bab7109157fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.104690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1003/1983] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:34 2022] DELETE /identity/v3/users/606e27c7d61546e1b546bab7109157fd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:35.110302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.115055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.116372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.141618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:35.143107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.211327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 981/1984] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:35.221842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:35.223593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.283004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abec36b-d16d-4b3b-9440-f7e5e4ff634c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:35.287654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:35.288265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.288546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.288820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.298763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:35.301228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:35.312337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1004/1985] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:35.386364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:35.387793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.401760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bb28c-1ad9-4c77-b3f4-3f11ea11106d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:35.407444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:35.408079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.408360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.408632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] PATH_INFO: `/v3/users/b08dad1bc98f4eb28b5e4d055707620d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.409993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08dad1bc98f4eb28b5e4d055707620d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:35.411896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:35.524627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:35.527567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.533999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08dad1bc98f4eb28b5e4d055707620d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.545208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d24d-8cbe-498f-b2e1-0f8b605f01fe service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:35.552146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:35.553313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.553781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.554200 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] PATH_INFO: `/v3/projects/24c3882ed5ac499f9aba5de4d43bf336` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.556420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c3882ed5ac499f9aba5de4d43bf336)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:35.559699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:35.561290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08dad1bc98f4eb28b5e4d055707620d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.583725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08dad1bc98f4eb28b5e4d055707620d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.584777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08dad1bc98f4eb28b5e4d055707620d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.585548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08dad1bc98f4eb28b5e4d055707620d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.614460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 982/1986] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:35 2022] DELETE /identity/v3/users/b08dad1bc98f4eb28b5e4d055707620d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:35.615629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:35.617521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.636355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c3882ed5ac499f9aba5de4d43bf336', 'request_id': 'req-a42717de-20d1-4ee4-a064-724c03516c47'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.642822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c3882ed5ac499f9aba5de4d43bf336', 'request_id': 'req-a42717de-20d1-4ee4-a064-724c03516c47'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.650243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c3882ed5ac499f9aba5de4d43bf336', 'request_id': 'req-a42717de-20d1-4ee4-a064-724c03516c47'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.666399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] The token cache is being invalidate because project 24c3882ed5ac499f9aba5de4d43bf336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:35.666681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.667723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a42717de-20d1-4ee4-a064-724c03516c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.673716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1005/1987] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:35 2022] DELETE /identity/v3/projects/24c3882ed5ac499f9aba5de4d43bf336 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:35.673716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22923f-68b6-48c6-817d-4c9f46eacb26 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.673716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22923f-68b6-48c6-817d-4c9f46eacb26 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.673716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22923f-68b6-48c6-817d-4c9f46eacb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.697038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cc22923f-68b6-48c6-817d-4c9f46eacb26 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:35.698897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cc22923f-68b6-48c6-817d-4c9f46eacb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.700567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:35.720940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df837e7-2842-4afc-987b-61e2f97d7fe9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:35.757436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:35.789500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1006/1988] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:35.791546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.792478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.794063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.800890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:35.801694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.802105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.802484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] PATH_INFO: `/v3/users/45d9b6bd4eea408db0f0b9df7fd7e89d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.803772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45d9b6bd4eea408db0f0b9df7fd7e89d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:35.805831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:35.823408 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:35.825141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.841866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1007/1989] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:35.841866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:35.842725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:35.864690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45d9b6bd4eea408db0f0b9df7fd7e89d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.873180 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45d9b6bd4eea408db0f0b9df7fd7e89d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.888546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45d9b6bd4eea408db0f0b9df7fd7e89d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.889099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45d9b6bd4eea408db0f0b9df7fd7e89d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.889762 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-2872e252-2105-469e-a577-f41910577b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45d9b6bd4eea408db0f0b9df7fd7e89d'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:35.910827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 983/1990] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:35 2022] DELETE /identity/v3/users/45d9b6bd4eea408db0f0b9df7fd7e89d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:35.912136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf923e-a41e-45c9-aa42-9e584790d25b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:35.912425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf923e-a41e-45c9-aa42-9e584790d25b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:35.912702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf923e-a41e-45c9-aa42-9e584790d25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:35.931804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026b440-dbd5-4c8f-8541-ed1dd2fe0dbb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:35.942528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-accf923e-a41e-45c9-aa42-9e584790d25b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:35.943760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-accf923e-a41e-45c9-aa42-9e584790d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.024326 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.025054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.025404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.025760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.032760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.072514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 984/1991] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:36.074054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037792f-647c-4278-b357-f05b7ccc3bcb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.074446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037792f-647c-4278-b357-f05b7ccc3bcb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.074821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037792f-647c-4278-b357-f05b7ccc3bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.112757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0037792f-647c-4278-b357-f05b7ccc3bcb None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:36.114208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0037792f-647c-4278-b357-f05b7ccc3bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.122110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.123589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.129771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 985/1992] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:36.131136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037792f-647c-4278-b357-f05b7ccc3bcb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.135164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1008/1993] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:36.143010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037792f-647c-4278-b357-f05b7ccc3bcb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.145837 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.148325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.149876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.150746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.151225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.151738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.152612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:36.153197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.154999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.160046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:36.165563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a582952-8fec-4956-b9a5-9314809b7f73 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.174423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.175818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.176154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.176434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.182889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 986/1994] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:36.183861 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.207231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.209092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.216038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce2c72-7442-4ef7-b0c7-d4e24b56fa4b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.223094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1009/1995] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:13:36.230078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.230994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.231280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.231554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.232692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.236244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.277566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 987/1996] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:36 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:36.282360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.293689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a47064-efaa-4ca1-a91c-4ed89c73b9fe service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.329095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:36.370319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.371005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.371280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.371549 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.372243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.374428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.401435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1010/1997] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:36 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:36.405059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.420129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0fbdd-d031-4057-8f28-0a33bb6ca3dd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.428700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.430474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.430755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.431029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.432184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.434035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.460107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 988/1998] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:36.461951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.474570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb014d0-4f61-4a97-b166-0bf7b4d78001 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.479769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ce41b-a48d-4606-8b83-53e38fb00592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.480485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ce41b-a48d-4606-8b83-53e38fb00592 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.480728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ce41b-a48d-4606-8b83-53e38fb00592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.481005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ce41b-a48d-4606-8b83-53e38fb00592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.484354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ce41b-a48d-4606-8b83-53e38fb00592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.486092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ce41b-a48d-4606-8b83-53e38fb00592 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.501672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1011/1999] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:36.503053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.516845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a2def-8cdf-45e1-adff-134bac20e714 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.536712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.537351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.537630 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.537947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] PATH_INFO: `/v3/projects/07cb899fe6934c27820d1329375e044d/users/3e4eccc7ece24cf4bcfee361d3342a4f/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.571501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cb899fe6934c27820d1329375e044d, user_id=3e4eccc7ece24cf4bcfee361d3342a4f, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.579192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.625174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 989/2000] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:36 2022] PUT /identity/v3/projects/07cb899fe6934c27820d1329375e044d/users/3e4eccc7ece24cf4bcfee361d3342a4f/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:36.630468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6234-30bc-494a-8ad1-db1e0df9b275 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.630790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6234-30bc-494a-8ad1-db1e0df9b275 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.631061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc6234-30bc-494a-8ad1-db1e0df9b275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.663979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-31fc6234-30bc-494a-8ad1-db1e0df9b275 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:36.665227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-31fc6234-30bc-494a-8ad1-db1e0df9b275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.668182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.670922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.765917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1012/2001] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:36.780513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e091a-e22b-4588-8268-3a1044d1d9a5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.784729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6234-30bc-494a-8ad1-db1e0df9b275 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.789485 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.791217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.791459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.791747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] PATH_INFO: `/v3/projects/97c52c65a3a347d0b9a0ca3852efaa46` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.793706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c52c65a3a347d0b9a0ca3852efaa46)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:36.795766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:36.862606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc6234-30bc-494a-8ad1-db1e0df9b275 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:36.884872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c52c65a3a347d0b9a0ca3852efaa46', 'request_id': 'req-00f8ddfa-0ca7-4418-a386-94ca86151871'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:36.891911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c52c65a3a347d0b9a0ca3852efaa46', 'request_id': 'req-00f8ddfa-0ca7-4418-a386-94ca86151871'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:36.900866 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c52c65a3a347d0b9a0ca3852efaa46', 'request_id': 'req-00f8ddfa-0ca7-4418-a386-94ca86151871'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:36.913069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] The token cache is being invalidate because project 97c52c65a3a347d0b9a0ca3852efaa46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:36.913314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:36.914021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:36.918452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 990/2002] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:36 2022] DELETE /identity/v3/projects/97c52c65a3a347d0b9a0ca3852efaa46 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:36.918452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:36.918452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.921392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:36.922469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:36.922740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:36.923012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:36.926527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:36.950283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:36.962366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:37.014751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8ddfa-0ca7-4418-a386-94ca86151871 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:37.025260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:37.026228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:37.026656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:37.027096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] PATH_INFO: `/v3/projects/36e5d228c4714f4a9baba0faa4c46d88` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:37.028421 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e5d228c4714f4a9baba0faa4c46d88)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:37.035379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:37.063557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:37.065534 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:37.079413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1013/2003] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:37.080533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:37.082850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:37.096148 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0754-d444-4ddf-8885-8f0a41258b83 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:37.108886 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e5d228c4714f4a9baba0faa4c46d88', 'request_id': 'req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.116541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e5d228c4714f4a9baba0faa4c46d88', 'request_id': 'req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.127389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e5d228c4714f4a9baba0faa4c46d88', 'request_id': 'req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.142785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] The token cache is being invalidate because project 36e5d228c4714f4a9baba0faa4c46d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:37.143373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.144939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.156070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 991/2004] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:36 2022] DELETE /identity/v3/projects/36e5d228c4714f4a9baba0faa4c46d88 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:37.164655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:37.166819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:37.195839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:37.197833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:37.197833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:37.197833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] PATH_INFO: `/v3/users/8608395ca01d464f9cc29dfff4c14b74` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:37.200018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8608395ca01d464f9cc29dfff4c14b74)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:37.202528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:37.277820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c234b-d5ca-4b3e-ac3f-b4994152ef92 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:37.287189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:37.287888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:37.288172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:37.288446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] PATH_INFO: `/v3/users/b041ecce24794c019742100f023c90bb` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:37.289821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b041ecce24794c019742100f023c90bb)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:37.295384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:37.320417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8608395ca01d464f9cc29dfff4c14b74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.334404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8608395ca01d464f9cc29dfff4c14b74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.359913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8608395ca01d464f9cc29dfff4c14b74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.375541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8608395ca01d464f9cc29dfff4c14b74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.377276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8608395ca01d464f9cc29dfff4c14b74'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.392357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1014/2005] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:37 2022] DELETE /identity/v3/users/8608395ca01d464f9cc29dfff4c14b74 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:37.416131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b041ecce24794c019742100f023c90bb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.436293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b041ecce24794c019742100f023c90bb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.448636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:37.470149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b041ecce24794c019742100f023c90bb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.471759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b041ecce24794c019742100f023c90bb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.472994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b041ecce24794c019742100f023c90bb'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:37.495764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 992/2006] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:37 2022] DELETE /identity/v3/users/b041ecce24794c019742100f023c90bb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:37.794668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:37.796310 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:37.848215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:37.876715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f2bc1-cbbc-4024-b992-ff6371d5f4a9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:37.936255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:37.936817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:37.937097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:37.937404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] PATH_INFO: `/v3/projects/cf5cf890a24e41aba06ab35ca359edc7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:37.938567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5cf890a24e41aba06ab35ca359edc7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:37.940685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:38.007451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5cf890a24e41aba06ab35ca359edc7', 'request_id': 'req-23c0150e-5cee-4f03-838a-676145fd3e5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.015170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5cf890a24e41aba06ab35ca359edc7', 'request_id': 'req-23c0150e-5cee-4f03-838a-676145fd3e5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.025609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5cf890a24e41aba06ab35ca359edc7', 'request_id': 'req-23c0150e-5cee-4f03-838a-676145fd3e5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.049086 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] The token cache is being invalidate because project cf5cf890a24e41aba06ab35ca359edc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:38.050019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.051018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.053089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1015/2007] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:37 2022] DELETE /identity/v3/projects/cf5cf890a24e41aba06ab35ca359edc7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:38.057478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:38.058777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:38.072382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f304c9-dbb7-4aeb-b1ba-ff0a3a18ac6e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:38.160854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:38.161684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:38.162234 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:38.162401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] PATH_INFO: `/v3/users/60d7e02a81ec451584e5cf1932d4a4dd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:38.163741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60d7e02a81ec451584e5cf1932d4a4dd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:38.166006 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:38.223460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60d7e02a81ec451584e5cf1932d4a4dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.234452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60d7e02a81ec451584e5cf1932d4a4dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.249885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60d7e02a81ec451584e5cf1932d4a4dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.250704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60d7e02a81ec451584e5cf1932d4a4dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.251437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-66ca8d59-68dc-4816-b249-abd564620c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60d7e02a81ec451584e5cf1932d4a4dd'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.266748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 993/2008] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:38 2022] DELETE /identity/v3/users/60d7e02a81ec451584e5cf1932d4a4dd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:38.339907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:38.341550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:38.360531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0150e-5cee-4f03-838a-676145fd3e5e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:38.476819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30785c94-a1f1-44db-afd2-0023d759cac3 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:38.477184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30785c94-a1f1-44db-afd2-0023d759cac3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:38.477465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30785c94-a1f1-44db-afd2-0023d759cac3 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:38.479696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 994/2009] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 14:13:38.512196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:38.513561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:38.513967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:38.514342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] PATH_INFO: `/v3/projects/55a983ac7f93459f879e02aaec6666b0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:38.517909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2404-0470-49ec-8d43-1da860259f22 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:38.518857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2404-0470-49ec-8d43-1da860259f22 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:38.519400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f2404-0470-49ec-8d43-1da860259f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:38.527880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a983ac7f93459f879e02aaec6666b0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:38.530471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:38.574793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:38.592713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ae1f2404-0470-49ec-8d43-1da860259f22 None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:38.595458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ae1f2404-0470-49ec-8d43-1da860259f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:38.649608 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a983ac7f93459f879e02aaec6666b0', 'request_id': 'req-16b21069-a62c-4cc8-a980-daae2fee7c16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.656282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a983ac7f93459f879e02aaec6666b0', 'request_id': 'req-16b21069-a62c-4cc8-a980-daae2fee7c16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.664559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a983ac7f93459f879e02aaec6666b0', 'request_id': 'req-16b21069-a62c-4cc8-a980-daae2fee7c16'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.686106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] The token cache is being invalidate because project 55a983ac7f93459f879e02aaec6666b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:38.689231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.690835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.694282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1016/2010] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:38 2022] DELETE /identity/v3/projects/55a983ac7f93459f879e02aaec6666b0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:38.695092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:38.699525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:38.726175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:38.745414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 995/2011] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:38.747268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:38.747268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:38.747510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:38.788964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] MFA Rules not processed for user `3e4eccc7ece24cf4bcfee361d3342a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:38.790406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:38.795981 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b21069-a62c-4cc8-a980-daae2fee7c16 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:38.803596 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:38.804238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:38.804783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:38.805061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] PATH_INFO: `/v3/users/e35dfe8442964bd793276386c2fe6f95` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:38.806824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35dfe8442964bd793276386c2fe6f95)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:38.808967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:38.882117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35dfe8442964bd793276386c2fe6f95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.890730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35dfe8442964bd793276386c2fe6f95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.906604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35dfe8442964bd793276386c2fe6f95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.915336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35dfe8442964bd793276386c2fe6f95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.916458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35dfe8442964bd793276386c2fe6f95'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:38.925312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1017/2012] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:38 2022] DELETE /identity/v3/users/e35dfe8442964bd793276386c2fe6f95 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:38.926263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:38.928001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:38.929493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 996/2013] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:38.930751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:38.932243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:38.967274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:39.015774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e035e3-0e04-44a9-871b-6889b79f5d00 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.021821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b20eb2-0a4a-4532-827f-7ef305a67377 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.023479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.024436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.024883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.025425 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] PATH_INFO: `/v3/users/2df89e85651a4801871a9edc96edb8af` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.027058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df89e85651a4801871a9edc96edb8af)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.029331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.089923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df89e85651a4801871a9edc96edb8af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.091349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.091970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.092303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.092725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.095288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.101085 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df89e85651a4801871a9edc96edb8af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.109039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.110885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.118798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df89e85651a4801871a9edc96edb8af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.125795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 997/2014] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:39.126839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.130909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.134211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df89e85651a4801871a9edc96edb8af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.135119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df89e85651a4801871a9edc96edb8af'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.149035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52ed5b-6fb8-4529-b847-49de491997c9 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.150748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1018/2015] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:38 2022] DELETE /identity/v3/users/2df89e85651a4801871a9edc96edb8af => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:39.153825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.157351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.169354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:39.231969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.233004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.233362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.233720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] PATH_INFO: `/v3/projects/8fb09e8fce2f4826b6b10ecf90808626` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.235372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb09e8fce2f4826b6b10ecf90808626)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.239257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.244566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ca1e-b559-4fa5-b1f0-d0ac1fb94452 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.250504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cfe6b-7b0e-4a23-a73d-6325db127421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.250635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cfe6b-7b0e-4a23-a73d-6325db127421 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.251558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cfe6b-7b0e-4a23-a73d-6325db127421 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.251947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cfe6b-7b0e-4a23-a73d-6325db127421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.254939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-324cfe6b-7b0e-4a23-a73d-6325db127421 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.302234 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb09e8fce2f4826b6b10ecf90808626', 'request_id': 'req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.314443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb09e8fce2f4826b6b10ecf90808626', 'request_id': 'req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.324999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:39.325988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb09e8fce2f4826b6b10ecf90808626', 'request_id': 'req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.328132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cfe6b-7b0e-4a23-a73d-6325db127421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.331096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cfe6b-7b0e-4a23-a73d-6325db127421 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.339857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] The token cache is being invalidate because project 8fb09e8fce2f4826b6b10ecf90808626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:39.340112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.340807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.342438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 998/2016] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:39 2022] DELETE /identity/v3/projects/8fb09e8fce2f4826b6b10ecf90808626 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:39.342994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1019/2017] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:39.343212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.344597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.355481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8018a-f262-4b24-8bb0-15807d0dcfe7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.355886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8018a-f262-4b24-8bb0-15807d0dcfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.356268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8018a-f262-4b24-8bb0-15807d0dcfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.380074 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-90f8018a-f262-4b24-8bb0-15807d0dcfe7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:39.381844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-90f8018a-f262-4b24-8bb0-15807d0dcfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.437280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fd840-b16f-48b2-8230-35cc4fcb7bf0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.443573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.444366 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.444884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.444996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] PATH_INFO: `/v3/projects/a66ebe0eb0ec44fa8fb3ed62d27f9599` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.446345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66ebe0eb0ec44fa8fb3ed62d27f9599)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.449551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.464867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1020/2018] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:39.470283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3018e5-c7cd-4c53-b260-2971c72f1053 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.470283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3018e5-c7cd-4c53-b260-2971c72f1053 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.470283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3018e5-c7cd-4c53-b260-2971c72f1053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.503757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ff3018e5-c7cd-4c53-b260-2971c72f1053 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:39.504933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff3018e5-c7cd-4c53-b260-2971c72f1053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.522100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:39.524843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66ebe0eb0ec44fa8fb3ed62d27f9599', 'request_id': 'req-5ae1070c-4033-4aed-9726-3ecd643c7b3b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.533463 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66ebe0eb0ec44fa8fb3ed62d27f9599', 'request_id': 'req-5ae1070c-4033-4aed-9726-3ecd643c7b3b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.544982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66ebe0eb0ec44fa8fb3ed62d27f9599', 'request_id': 'req-5ae1070c-4033-4aed-9726-3ecd643c7b3b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.562567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] The token cache is being invalidate because project a66ebe0eb0ec44fa8fb3ed62d27f9599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:39.562864 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.563548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.569316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 999/2019] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:39 2022] DELETE /identity/v3/projects/a66ebe0eb0ec44fa8fb3ed62d27f9599 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:39.569316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.569316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.599129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1021/2020] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:39.600366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266540c-842a-401b-8845-3ee736021706 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.600655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266540c-842a-401b-8845-3ee736021706 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.600932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266540c-842a-401b-8845-3ee736021706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.622449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8266540c-842a-401b-8845-3ee736021706 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:39.624075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8266540c-842a-401b-8845-3ee736021706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.640912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1022/2021] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:39.640912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266540c-842a-401b-8845-3ee736021706 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.646644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266540c-842a-401b-8845-3ee736021706 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.649221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1070c-4033-4aed-9726-3ecd643c7b3b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.652271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa9a26-94ad-4aa3-b6c9-a443d9edcea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.652953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9a26-94ad-4aa3-b6c9-a443d9edcea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.653302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9a26-94ad-4aa3-b6c9-a443d9edcea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.653647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9a26-94ad-4aa3-b6c9-a443d9edcea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.654413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9a26-94ad-4aa3-b6c9-a443d9edcea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:39.654765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.654897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9a26-94ad-4aa3-b6c9-a443d9edcea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.655400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.655684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.655960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] PATH_INFO: `/v3/projects/2a67c5e89219434689d202cf30feeeb7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.656789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9a26-94ad-4aa3-b6c9-a443d9edcea5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.657139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a67c5e89219434689d202cf30feeeb7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.670634 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1023/2022] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:39.671967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.672226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0342e-da44-45bf-856e-6b0376487017 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.672615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0342e-da44-45bf-856e-6b0376487017 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.672994 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0342e-da44-45bf-856e-6b0376487017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.704206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ebc0342e-da44-45bf-856e-6b0376487017 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:39.705906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ebc0342e-da44-45bf-856e-6b0376487017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.717088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1024/2023] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:39.718496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0342e-da44-45bf-856e-6b0376487017 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.730623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0342e-da44-45bf-856e-6b0376487017 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.738032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a67c5e89219434689d202cf30feeeb7', 'request_id': 'req-dd763383-1a63-4473-8013-ee4b8e7458af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.745027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a67c5e89219434689d202cf30feeeb7', 'request_id': 'req-dd763383-1a63-4473-8013-ee4b8e7458af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.754591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a67c5e89219434689d202cf30feeeb7', 'request_id': 'req-dd763383-1a63-4473-8013-ee4b8e7458af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.768671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] The token cache is being invalidate because project 2a67c5e89219434689d202cf30feeeb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:39.768969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.769641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:39.771760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1000/2024] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:39 2022] DELETE /identity/v3/projects/2a67c5e89219434689d202cf30feeeb7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:39.772399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.775784 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.794964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fda4f8-a8c1-46ae-826a-dd81ad4d7d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.795595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fda4f8-a8c1-46ae-826a-dd81ad4d7d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.795871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fda4f8-a8c1-46ae-826a-dd81ad4d7d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.796141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fda4f8-a8c1-46ae-826a-dd81ad4d7d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.796765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fda4f8-a8c1-46ae-826a-dd81ad4d7d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.798716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fda4f8-a8c1-46ae-826a-dd81ad4d7d66 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.817093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:39.831946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1025/2025] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:39.833093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.833380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.833663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.851555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd763383-1a63-4473-8013-ee4b8e7458af admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.853850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:39.855162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.860345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:39.912354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b556860-2706-4655-8066-3a6ae4d784d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.913257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556860-2706-4655-8066-3a6ae4d784d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.913687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556860-2706-4655-8066-3a6ae4d784d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.914161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b556860-2706-4655-8066-3a6ae4d784d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.915159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b556860-2706-4655-8066-3a6ae4d784d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:39.915774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b556860-2706-4655-8066-3a6ae4d784d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.918387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b556860-2706-4655-8066-3a6ae4d784d4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.920821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1026/2026] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:39.921873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.929072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.940467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1001/2027] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:39.942282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6d9ff-02de-4944-b3da-1bf411ad5da8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.942578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6d9ff-02de-4944-b3da-1bf411ad5da8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.942684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e77e-e0e6-4a92-824e-7073722b46cb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.942862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6d9ff-02de-4944-b3da-1bf411ad5da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.948508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.949558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.949870 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.950142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.950773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.952822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.963906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7ee6d9ff-02de-4944-b3da-1bf411ad5da8 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:39.965141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7ee6d9ff-02de-4944-b3da-1bf411ad5da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:39.975356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1002/2028] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:39.975972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6d9ff-02de-4944-b3da-1bf411ad5da8 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.980471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1027/2029] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/users => generated 293 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:39.980471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:39.985666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6d9ff-02de-4944-b3da-1bf411ad5da8 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.991432 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce5bc1-7930-4091-b880-081c3075dd77 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:39.991619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.992239 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.992513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.992794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.993481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:39.995591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:39.996657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:39.997331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:39.997710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:39.999154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:39.999861 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:40.000232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.002327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.016247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1028/2030] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:40.018296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.030585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f567186-36f7-4c7d-8231-9099e4ae0a23 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.031316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1003/2031] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:39 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:40.032254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.101132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.101851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.102213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.102633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.103448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.105238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.108680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7c1f2-1cc2-41f0-a661-e28e7322997c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.115178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.115787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.116098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.116376 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.117007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.118985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.122860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1029/2032] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:40.122860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.132004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e5d62-8958-4f74-9a4a-31693bbb871c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.137039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.137680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.138076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.138449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.139097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.141273 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.148046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1004/2033] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:40 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:40.148840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.167014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1030/2034] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:40 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:40.167014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.177497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5db3e-1ea1-4cbd-b328-1a9315a89161 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.216313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453bac4-86c3-4705-8c38-4d6d4d73c70b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.222977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.223283 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:40.224149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.224605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.225244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.225965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.227614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.234357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:40.240429 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.243916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1005/2035] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:40.244054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.244054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.244054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.244054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.244364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.246699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.257830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1911a5d-1b6d-4474-81a6-098dc60c5d78 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.262964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.263561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.263838 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.264108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.264743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.266361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.269783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1031/2036] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:40 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:40.269783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.278525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64137a4f-3808-451f-8c85-034f88098e3e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.282575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1006/2037] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:40.283397 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.283512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.284366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.284849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.285128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] PATH_INFO: `/v3/projects/8eecd1cf25b9456c87d90f3182998d4d/users/d13118652ef741be96ed0423c3c75768/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.293296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb564a-bb2f-448e-b583-ec0ffc98c7b7 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.298168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.298755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.299037 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.299850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.300843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.301187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eecd1cf25b9456c87d90f3182998d4d, user_id=d13118652ef741be96ed0423c3c75768, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.302478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.303843 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.316616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1007/2038] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:40.316616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.318407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1032/2039] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:40 2022] PUT /identity/v3/projects/8eecd1cf25b9456c87d90f3182998d4d/users/d13118652ef741be96ed0423c3c75768/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:40.319220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.333160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc065ab-db9b-4c5b-9bdb-02d105b4045f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.382916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef16e6-23c6-4c71-81a6-8de019eaedd9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.387677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaeb028-141f-4e74-93d1-8bc0780f1ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.388312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb028-141f-4e74-93d1-8bc0780f1ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.388596 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb028-141f-4e74-93d1-8bc0780f1ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.388925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaeb028-141f-4e74-93d1-8bc0780f1ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.389568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaeb028-141f-4e74-93d1-8bc0780f1ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.390885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.391225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaeb028-141f-4e74-93d1-8bc0780f1ed4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.391490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.391771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.392042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.392677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.394299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.401906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1033/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:40.403124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.403409 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.403685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.407476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1008/2041] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:40.407476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.416928 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c468f91-f852-4d66-9a2b-5f5b1713e989 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.421645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.422341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.422604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.422881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] PATH_INFO: `/v3/projects/e77dc690fa764e61b0a021fe46a99cbd/users/e6c9ed60880745f9adab0e0c635674f9/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.433668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] MFA Rules not processed for user `d13118652ef741be96ed0423c3c75768`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:40.434927 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:40.438328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77dc690fa764e61b0a021fe46a99cbd, user_id=e6c9ed60880745f9adab0e0c635674f9, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.442215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.457227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1009/2042] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:40 2022] PUT /identity/v3/projects/e77dc690fa764e61b0a021fe46a99cbd/users/e6c9ed60880745f9adab0e0c635674f9/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:40.458062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.514889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1034/2043] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:40.529362 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.531157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:40.550872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baf900-f3f1-42a6-a2ab-e60d73447196 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.556107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22315030-0de1-46d3-8a09-5d44bf3bfc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.556775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315030-0de1-46d3-8a09-5d44bf3bfc73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.557058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315030-0de1-46d3-8a09-5d44bf3bfc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.557334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22315030-0de1-46d3-8a09-5d44bf3bfc73 admin admin] PATH_INFO: `/v3/projects/8bb225cd095d4e8bba029946f72fd02a/users/0b6ae205411c418d83ede369929d2333/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.572967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:40.573593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315030-0de1-46d3-8a09-5d44bf3bfc73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb225cd095d4e8bba029946f72fd02a, user_id=0b6ae205411c418d83ede369929d2333, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.576598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22315030-0de1-46d3-8a09-5d44bf3bfc73 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.590374 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1010/2044] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:40 2022] PUT /identity/v3/projects/8bb225cd095d4e8bba029946f72fd02a/users/0b6ae205411c418d83ede369929d2333/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:40.591767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdef890-6868-4a65-8497-fe847ccecd22 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.592056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdef890-6868-4a65-8497-fe847ccecd22 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.592330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdef890-6868-4a65-8497-fe847ccecd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.595056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304496d4-c656-43ca-bdd0-1dc5b54a4d7a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.621473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ebdef890-6868-4a65-8497-fe847ccecd22 None None] MFA Rules not processed for user `e6c9ed60880745f9adab0e0c635674f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:40.623162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ebdef890-6868-4a65-8497-fe847ccecd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:40.645709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:40.657998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.658637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.659107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.659466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.692134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1011/2045] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:40.695094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7ce0-400a-4bb8-82c3-ffcc5c9d5266 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.695505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7ce0-400a-4bb8-82c3-ffcc5c9d5266 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.695785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7ce0-400a-4bb8-82c3-ffcc5c9d5266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.716581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.717805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2c1a7ce0-400a-4bb8-82c3-ffcc5c9d5266 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:40.718126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.719011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2c1a7ce0-400a-4bb8-82c3-ffcc5c9d5266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:40.727688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1035/2046] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:40.734619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.736093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:40.800369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b274ae-7f0a-4c62-a98a-e483d8d3ec64 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.804091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7214e6addc646a899efb82283ededc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.804686 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.804959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.805224 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.810974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1012/2047] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:40.818153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.819787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.823320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15edb-90dc-42b0-a388-ae6c86d64625 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.823659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15edb-90dc-42b0-a388-ae6c86d64625 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.823940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f15edb-90dc-42b0-a388-ae6c86d64625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.830267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1036/2048] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:40.831832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:40.833203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:40.854769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-71f15edb-90dc-42b0-a388-ae6c86d64625 None None] MFA Rules not processed for user `3e4eccc7ece24cf4bcfee361d3342a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:40.855981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-71f15edb-90dc-42b0-a388-ae6c86d64625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:40.910190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:40.915497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecdca8c-b475-4432-8ca8-0bf5cd64e0b8 service glance] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:40.920501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:40.921127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:40.921434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:40.921801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:40.929880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1013/2049] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:40.934233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:40.936303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:40.947318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1037/2050] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:42.043003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70716410-0dce-4b56-99ad-e6e93c7d0f16 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:42.043577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70716410-0dce-4b56-99ad-e6e93c7d0f16 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:42.043577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70716410-0dce-4b56-99ad-e6e93c7d0f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:42.094887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-70716410-0dce-4b56-99ad-e6e93c7d0f16 None None] MFA Rules not processed for user `0b6ae205411c418d83ede369929d2333`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:42.096947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-70716410-0dce-4b56-99ad-e6e93c7d0f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:42.198541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:42.202998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1014/2051] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:42.208573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:42.218689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e378c4-a7cc-43b7-9c99-14202a88303f service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:42.225527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:42.225908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:42.226292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:42.226665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:42.227261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:42.229183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:42.265821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1038/2052] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:42 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:42.267389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70716410-0dce-4b56-99ad-e6e93c7d0f16 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:42.287373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70716410-0dce-4b56-99ad-e6e93c7d0f16 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:42.364887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:42.365521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:42.365845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:42.366150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:42.366939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:42.369193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:42.401275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1015/2053] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:42 2022] POST /identity/v3/users => generated 309 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:42.402771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:42.417980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af0d7-ec40-4364-9086-620a09cc6216 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:42.424185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:42.425372 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:42.426199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:42.426670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:42.427381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:42.428992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:42.465798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1039/2054] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:42.469059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:42.482380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e986753-fe4f-4141-93d6-713219e371b0 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:42.489493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:42.490702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:42.491804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:42.492257 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] PATH_INFO: `/v3/projects/7029537938404e7b8c54a9e744254a3b/users/51586648a0df4106a69368562754d785/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:42.559883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7029537938404e7b8c54a9e744254a3b, user_id=51586648a0df4106a69368562754d785, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:42.568010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:42.583457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1016/2055] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:42 2022] PUT /identity/v3/projects/7029537938404e7b8c54a9e744254a3b/users/51586648a0df4106a69368562754d785/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:42.587842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:42.604727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26b8e6-5a72-4b81-addc-686aff0cc986 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:42.705995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a6813-1718-417e-b1bb-84bc9ab5cbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:42.706605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6813-1718-417e-b1bb-84bc9ab5cbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:42.706903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6813-1718-417e-b1bb-84bc9ab5cbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:42.707152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6813-1718-417e-b1bb-84bc9ab5cbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:42.707789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a6813-1718-417e-b1bb-84bc9ab5cbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:42.709514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a6813-1718-417e-b1bb-84bc9ab5cbd6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:42.719995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1040/2056] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:42.724966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:42.737271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536c384-fdfb-43f3-be87-65e4ef95da39 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:42.742512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:42.743186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:42.743474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:42.743783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] PATH_INFO: `/v3/projects/7029537938404e7b8c54a9e744254a3b/users/51586648a0df4106a69368562754d785/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:42.746073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7029537938404e7b8c54a9e744254a3b, user_id=51586648a0df4106a69368562754d785, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:42.748921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:42.760880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1017/2057] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:42 2022] PUT /identity/v3/projects/7029537938404e7b8c54a9e744254a3b/users/51586648a0df4106a69368562754d785/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:43.463584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2726d5-5163-498e-b65b-baa1b9c06458 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:43.465777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2726d5-5163-498e-b65b-baa1b9c06458 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:43.467222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2726d5-5163-498e-b65b-baa1b9c06458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:43.496260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:43.531401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4e2726d5-5163-498e-b65b-baa1b9c06458 None None] MFA Rules not processed for user `51586648a0df4106a69368562754d785`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:43.533174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4e2726d5-5163-498e-b65b-baa1b9c06458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:43.680889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce864e0-327b-4dde-82a1-6c3d82404353 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:43.693084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:43.694147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:43.694675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:43.695146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] PATH_INFO: `/v3/users/d13118652ef741be96ed0423c3c75768` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:43.696534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13118652ef741be96ed0423c3c75768)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:43.698921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:43.708605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1041/2058] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:43.802166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13118652ef741be96ed0423c3c75768'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:43.808824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:43.824166 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13118652ef741be96ed0423c3c75768'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:43.850796 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13118652ef741be96ed0423c3c75768'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:43.857495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13118652ef741be96ed0423c3c75768'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:43.858522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13118652ef741be96ed0423c3c75768'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:43.886171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1018/2059] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:43 2022] DELETE /identity/v3/users/d13118652ef741be96ed0423c3c75768 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:43.893437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f809a5-f1da-4c09-8872-63b0faad34e6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:43.894019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f809a5-f1da-4c09-8872-63b0faad34e6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:43.894514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f809a5-f1da-4c09-8872-63b0faad34e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:43.922705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-85f809a5-f1da-4c09-8872-63b0faad34e6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:43.924029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-85f809a5-f1da-4c09-8872-63b0faad34e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:43.977467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:44.013955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1042/2060] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:44.035589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:44.038440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:44.136672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb0430-fe58-43fb-833d-bd21b761c7c3 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:44.141422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011017c7-c358-4643-add0-3548d405c63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:44.142158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011017c7-c358-4643-add0-3548d405c63f service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.142508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011017c7-c358-4643-add0-3548d405c63f service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.142916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011017c7-c358-4643-add0-3548d405c63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.157942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011017c7-c358-4643-add0-3548d405c63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:44.159935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011017c7-c358-4643-add0-3548d405c63f service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:44.175210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1019/2061] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:13:44.400984 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc79a0-ba5e-4d43-a57f-f17a539802f8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.401289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc79a0-ba5e-4d43-a57f-f17a539802f8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.402010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc79a0-ba5e-4d43-a57f-f17a539802f8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.403822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1043/2062] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:13:44.416144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5c064-8b68-4764-a527-2eb1dd321fe8 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.416144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5c064-8b68-4764-a527-2eb1dd321fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.416144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5c064-8b68-4764-a527-2eb1dd321fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.438481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-79f5c064-8b68-4764-a527-2eb1dd321fe8 None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:44.440189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-79f5c064-8b68-4764-a527-2eb1dd321fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:44.565541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1020/2063] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:44.594898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36f91d-1be6-4176-b70e-73929f782612 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.595422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36f91d-1be6-4176-b70e-73929f782612 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.595523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36f91d-1be6-4176-b70e-73929f782612 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.596813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1044/2064] 10.222.0.20 () {54 vars in 961 bytes} [Tue Sep 13 14:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:13:44.602153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.602441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.602714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.628981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] MFA Rules not processed for user `d09c0b02c24f4c5db5ed35fdee2589b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:44.630439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:44.642589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1021/2065] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Sep 13 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:44.651715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36f91d-1be6-4176-b70e-73929f782612 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:44.669851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36f91d-1be6-4176-b70e-73929f782612 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:44.680570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf5e7d-3c47-48f7-a8dc-6a7e12c74b5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:44.681876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5e7d-3c47-48f7-a8dc-6a7e12c74b5d service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.682267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5e7d-3c47-48f7-a8dc-6a7e12c74b5d service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.682633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf5e7d-3c47-48f7-a8dc-6a7e12c74b5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.695970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf5e7d-3c47-48f7-a8dc-6a7e12c74b5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:44.702337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf5e7d-3c47-48f7-a8dc-6a7e12c74b5d service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:44.723565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1045/2066] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:44.775986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:44.777449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:44.803676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f8444-535b-4820-b849-aad71bf9e349 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:44.809888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:44.811161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.811594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.812035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] PATH_INFO: `/v3/projects/8eecd1cf25b9456c87d90f3182998d4d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.813351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eecd1cf25b9456c87d90f3182998d4d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:44.816169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:44.879799 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eecd1cf25b9456c87d90f3182998d4d', 'request_id': 'req-8d066ef3-331d-40fe-9e7e-55f27ea4d903'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:44.888640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:44.891523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eecd1cf25b9456c87d90f3182998d4d', 'request_id': 'req-8d066ef3-331d-40fe-9e7e-55f27ea4d903'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:44.903787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eecd1cf25b9456c87d90f3182998d4d', 'request_id': 'req-8d066ef3-331d-40fe-9e7e-55f27ea4d903'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:44.922976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] The token cache is being invalidate because project 8eecd1cf25b9456c87d90f3182998d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:44.923280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:44.924123 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8d066ef3-331d-40fe-9e7e-55f27ea4d903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:44.927128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1022/2067] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:44 2022] DELETE /identity/v3/projects/8eecd1cf25b9456c87d90f3182998d4d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:44.932330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53738eea-bbc6-4ab4-b86b-79fa021cb88c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:44.932702 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53738eea-bbc6-4ab4-b86b-79fa021cb88c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:44.933049 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53738eea-bbc6-4ab4-b86b-79fa021cb88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:44.965423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-53738eea-bbc6-4ab4-b86b-79fa021cb88c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:44.967661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-53738eea-bbc6-4ab4-b86b-79fa021cb88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.047876 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1046/2068] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:45.055833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.056141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.056412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.084705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:45.086370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.096356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1023/2069] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:45.100939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53738eea-bbc6-4ab4-b86b-79fa021cb88c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.132124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53738eea-bbc6-4ab4-b86b-79fa021cb88c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.134233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.136234 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.157971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.159051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.161400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.161818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.162194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc240f-ff04-4e3d-b128-6843d4ba580e None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.164305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:45.164691 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.169196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.184179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.185005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.185378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.185706 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] PATH_INFO: `/v3/users/0b6ae205411c418d83ede369929d2333` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.187701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6ae205411c418d83ede369929d2333)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.191218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.193717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1047/2070] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:45.198588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.213545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5f3e6-51f5-4daf-b218-42756f2d3eb6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.232579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.233417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.233703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.234012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.234651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.237394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.265919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6ae205411c418d83ede369929d2333'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.278345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6ae205411c418d83ede369929d2333'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.282648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1048/2071] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:45 2022] POST /identity/v3/projects => generated 391 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:45.287265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.302157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6ae205411c418d83ede369929d2333'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.303355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6ae205411c418d83ede369929d2333'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.304182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6ae205411c418d83ede369929d2333'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.326450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1024/2072] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:45 2022] DELETE /identity/v3/users/0b6ae205411c418d83ede369929d2333 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:45.384755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261bede-365c-41c5-a4dc-3898e0513c1b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.451066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:45.481922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.482848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.484160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.484440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.485262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.488499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.522150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1049/2073] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:45 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:45.526062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.534437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.561582 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc2423-8270-4b61-8862-f10d4fc86304 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.563873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.564062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.564665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.565035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.565379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.566223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.566970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.568441 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.582437 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37138246-8d41-4a5e-9f6c-bdbcd5cc3873 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.594117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.594878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1025/2074] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:45.596920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.597369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.597463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] PATH_INFO: `/v3/projects/8bb225cd095d4e8bba029946f72fd02a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.599597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.601617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb225cd095d4e8bba029946f72fd02a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.603733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.610742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974dc580-fb98-413d-baea-31422f72206c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.616622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.617377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.617793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.618328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.622392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.624326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.637065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1026/2075] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:45.640892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.653201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31693e70-e783-465b-be6a-80d1f7e4a4ba admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.670614 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb225cd095d4e8bba029946f72fd02a', 'request_id': 'req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.677963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb225cd095d4e8bba029946f72fd02a', 'request_id': 'req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.687368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb225cd095d4e8bba029946f72fd02a', 'request_id': 'req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.706179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] The token cache is being invalidate because project 8bb225cd095d4e8bba029946f72fd02a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:45.707229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.708411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:45.712835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1050/2076] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:45 2022] DELETE /identity/v3/projects/8bb225cd095d4e8bba029946f72fd02a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:45.713951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.715299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.726379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb4f30-b72f-427d-99c6-aff5eee36d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.726379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4f30-b72f-427d-99c6-aff5eee36d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.726379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4f30-b72f-427d-99c6-aff5eee36d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.726379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4f30-b72f-427d-99c6-aff5eee36d8a admin admin] PATH_INFO: `/v3/projects/674ceeddd4ed4a0fbc181bfeea2a0f30/users/31bf07c8b8414a66be9dbe0f0035d364/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.741458 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb4f30-b72f-427d-99c6-aff5eee36d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=674ceeddd4ed4a0fbc181bfeea2a0f30, user_id=31bf07c8b8414a66be9dbe0f0035d364, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.744132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb4f30-b72f-427d-99c6-aff5eee36d8a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.767210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1027/2077] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:45 2022] PUT /identity/v3/projects/674ceeddd4ed4a0fbc181bfeea2a0f30/users/31bf07c8b8414a66be9dbe0f0035d364/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:45.780267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.781709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.782149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.807225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec086d8f-066f-4f3e-ac0e-d98f87d191b8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:45.815637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:45.837361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] MFA Rules not processed for user `31bf07c8b8414a66be9dbe0f0035d364`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:45.838601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.903421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:45.904059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:45.904337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:45.904609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] PATH_INFO: `/v3/users/51586648a0df4106a69368562754d785` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:45.906297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51586648a0df4106a69368562754d785)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:45.908739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:45.920838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:45.935525 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1028/2078] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:45.946718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:45.948098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:45.991024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51586648a0df4106a69368562754d785'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.004319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51586648a0df4106a69368562754d785'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.020138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51586648a0df4106a69368562754d785'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.021395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57846279-2d4f-42f2-bbdf-4c44af200914 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.025528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9106b-5b7b-479c-8b80-f980adb6491f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.026280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9106b-5b7b-479c-8b80-f980adb6491f service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.026558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9106b-5b7b-479c-8b80-f980adb6491f service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.026860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9106b-5b7b-479c-8b80-f980adb6491f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.033311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51586648a0df4106a69368562754d785'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.034509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51586648a0df4106a69368562754d785'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.037641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9106b-5b7b-479c-8b80-f980adb6491f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.039146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9106b-5b7b-479c-8b80-f980adb6491f service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.041583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1051/2079] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:45 2022] DELETE /identity/v3/users/51586648a0df4106a69368562754d785 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:46.051938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1029/2080] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:13:46.237760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:46.240854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:46.354435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7fa9-8ecc-412d-aa33-54988f4905b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.371753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.372545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.373970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.374354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] PATH_INFO: `/v3/projects/7029537938404e7b8c54a9e744254a3b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.378487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7029537938404e7b8c54a9e744254a3b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.380523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.462529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7029537938404e7b8c54a9e744254a3b', 'request_id': 'req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.469659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7029537938404e7b8c54a9e744254a3b', 'request_id': 'req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.481940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7029537938404e7b8c54a9e744254a3b', 'request_id': 'req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.501046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] The token cache is being invalidate because project 7029537938404e7b8c54a9e744254a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:46.501361 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.502007 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a5c6eb23-00fd-46d1-9ba0-de51db364cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:46.503747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1052/2081] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:46 2022] DELETE /identity/v3/projects/7029537938404e7b8c54a9e744254a3b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:46.508342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6723-8050-4714-bcfb-33d96da411bc None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.508636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6723-8050-4714-bcfb-33d96da411bc None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.508938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6723-8050-4714-bcfb-33d96da411bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.534416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-66ef6723-8050-4714-bcfb-33d96da411bc None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:46.535835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-66ef6723-8050-4714-bcfb-33d96da411bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:46.601853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1030/2082] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:46.608950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b9113-55b2-443c-9516-f10b51857ba4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.609438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b9113-55b2-443c-9516-f10b51857ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.609533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b9113-55b2-443c-9516-f10b51857ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.633069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b06b9113-55b2-443c-9516-f10b51857ba4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:46.634399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b06b9113-55b2-443c-9516-f10b51857ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:46.642241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1053/2083] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:46.646980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef6723-8050-4714-bcfb-33d96da411bc None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:46.656855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef6723-8050-4714-bcfb-33d96da411bc None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.662077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.662670 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.662950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.663222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.663890 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:46.664189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.665794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.677842 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1031/2084] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:13:46.682084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b9113-55b2-443c-9516-f10b51857ba4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:46.686604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:46.692988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b9113-55b2-443c-9516-f10b51857ba4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.698729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.699324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.699665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.699940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.700622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.702624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.734784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1054/2085] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:46 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:46.735828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:46.746556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2014eb9-c9ab-4742-beb8-01102bd74c94 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.776422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:46.805788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.806424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.806703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.806976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.807620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.809564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.831733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1032/2086] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:46 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:46.835860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:46.847855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1109-e97e-4840-ab00-22094f8e23c1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.853059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.853568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.853940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.854218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.854893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.856542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.881089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1055/2087] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:46.884962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:46.897086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf7a50-330d-487a-8d83-30230fbbfd9a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.902328 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47257c2b-cbb5-4423-b385-c82b5f5402d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.902926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47257c2b-cbb5-4423-b385-c82b5f5402d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.903233 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47257c2b-cbb5-4423-b385-c82b5f5402d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.903510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47257c2b-cbb5-4423-b385-c82b5f5402d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.904185 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47257c2b-cbb5-4423-b385-c82b5f5402d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.905944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47257c2b-cbb5-4423-b385-c82b5f5402d3 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.918764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1033/2088] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:46.923129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:46.933104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90271d1a-36a4-4c87-a1e7-0765da4e78f3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:46.937605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:46.938336 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.938619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.939070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] PATH_INFO: `/v3/projects/7a2f245a8aac4b76a8fbb2eba1386917/users/0fac2d044cbf461ab7adb3442e2d2341/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:46.956705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a2f245a8aac4b76a8fbb2eba1386917, user_id=0fac2d044cbf461ab7adb3442e2d2341, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:46.959533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:46.973136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1056/2089] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:46 2022] PUT /identity/v3/projects/7a2f245a8aac4b76a8fbb2eba1386917/users/0fac2d044cbf461ab7adb3442e2d2341/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:46.983338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473b8ef-c409-460c-9365-95b087dbe350 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:46.983626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473b8ef-c409-460c-9365-95b087dbe350 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:46.983931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473b8ef-c409-460c-9365-95b087dbe350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.012790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3473b8ef-c409-460c-9365-95b087dbe350 None None] MFA Rules not processed for user `0fac2d044cbf461ab7adb3442e2d2341`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:47.013974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3473b8ef-c409-460c-9365-95b087dbe350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:47.081459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1034/2090] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:47.086137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:47.149403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87a524-1cd0-424b-b378-80c55a14fb8b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:47.154065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:47.154661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.154936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.155202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.155824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:47.157696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:47.187062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1057/2091] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:47 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:47.188399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473b8ef-c409-460c-9365-95b087dbe350 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:47.199701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473b8ef-c409-460c-9365-95b087dbe350 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:47.257119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:47.257772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.258100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.258373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.259010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:47.260995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:47.283087 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1035/2092] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:47 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:47.287177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:47.297377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd221b8a-973c-4cb6-8c49-7092e0ce9cca admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:47.302282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:47.302892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.303172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.303444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.304093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:47.306331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:47.318270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1058/2093] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:47.322495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:47.336521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b33d5-137a-47fb-9251-0dc329bdf71c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:47.341316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:47.342011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.342294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.342568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] PATH_INFO: `/v3/projects/97209e4ecab24cd9ae352bd2f2b76ff8/users/5fe9e621590b465188168929afc59bac/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.359590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97209e4ecab24cd9ae352bd2f2b76ff8, user_id=5fe9e621590b465188168929afc59bac, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:47.362628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:47.376077 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1036/2094] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:47 2022] PUT /identity/v3/projects/97209e4ecab24cd9ae352bd2f2b76ff8/users/5fe9e621590b465188168929afc59bac/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:47.380677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:47.392245 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ea697-44ae-4fc6-bcf2-9cafdaab8d61 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:47.461912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49836a23-bdb8-4968-bff4-abf39c35d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:47.462590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49836a23-bdb8-4968-bff4-abf39c35d5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.462878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49836a23-bdb8-4968-bff4-abf39c35d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.463154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49836a23-bdb8-4968-bff4-abf39c35d5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.463811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49836a23-bdb8-4968-bff4-abf39c35d5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:47.465621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49836a23-bdb8-4968-bff4-abf39c35d5ac admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:47.480354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1059/2095] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:47.481558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:47.489333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:47.493973 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed19c3-f4de-4aa3-89ae-9f39c647ceb2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:47.500999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:47.502345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.502967 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.503708 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] PATH_INFO: `/v3/projects/97209e4ecab24cd9ae352bd2f2b76ff8/users/5fe9e621590b465188168929afc59bac/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.505592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97209e4ecab24cd9ae352bd2f2b76ff8, user_id=5fe9e621590b465188168929afc59bac, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:47.509795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:47.520535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1037/2096] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:47 2022] PUT /identity/v3/projects/97209e4ecab24cd9ae352bd2f2b76ff8/users/5fe9e621590b465188168929afc59bac/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:47.537443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c347473-9888-45cd-9dc9-b5a6d14e91f9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.537758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c347473-9888-45cd-9dc9-b5a6d14e91f9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.538105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c347473-9888-45cd-9dc9-b5a6d14e91f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.596980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3c347473-9888-45cd-9dc9-b5a6d14e91f9 None None] MFA Rules not processed for user `5fe9e621590b465188168929afc59bac`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:47.598598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3c347473-9888-45cd-9dc9-b5a6d14e91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:47.607725 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:47.720466 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1060/2097] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:47.728574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:47.730454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:47.808325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233d567-2b0e-4e84-aeb3-e66ff95a6224 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:47.813649 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4ad81-9163-47cc-bcf8-29956527aeb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:47.814423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4ad81-9163-47cc-bcf8-29956527aeb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:47.814783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4ad81-9163-47cc-bcf8-29956527aeb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:47.815149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4ad81-9163-47cc-bcf8-29956527aeb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:47.824858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4ad81-9163-47cc-bcf8-29956527aeb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:47.826346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4ad81-9163-47cc-bcf8-29956527aeb2 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:47.847643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1038/2098] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:49.297947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c347473-9888-45cd-9dc9-b5a6d14e91f9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:49.361433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c347473-9888-45cd-9dc9-b5a6d14e91f9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:49.365926 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:49.366548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:49.366826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:49.367098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] PATH_INFO: `/v3/users/0fac2d044cbf461ab7adb3442e2d2341` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:49.368228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fac2d044cbf461ab7adb3442e2d2341)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:49.370243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:49.417033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fac2d044cbf461ab7adb3442e2d2341'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:49.424216 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fac2d044cbf461ab7adb3442e2d2341'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:49.437469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fac2d044cbf461ab7adb3442e2d2341'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:49.445905 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fac2d044cbf461ab7adb3442e2d2341'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:49.446580 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fac2d044cbf461ab7adb3442e2d2341'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:49.452995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1061/2099] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:49 2022] DELETE /identity/v3/users/0fac2d044cbf461ab7adb3442e2d2341 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:49.458816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:49.459309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:49.459475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:49.478658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:49.479851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:49.539570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1039/2100] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:49.549657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:49.551140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:49.574785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:49.613873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbeea1-36ed-4e20-9f7a-8b3b06f7b70c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:49.617522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:49.618415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:49.618693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:49.619286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:49.629650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:49.631118 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:49.640758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1062/2101] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:13:49.927792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:49.929342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:49.947594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387dfef-630c-45a3-8100-ddaa99e11da4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:49.956324 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:49.957084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:49.957636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:49.957825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] PATH_INFO: `/v3/projects/7a2f245a8aac4b76a8fbb2eba1386917` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:49.959072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a2f245a8aac4b76a8fbb2eba1386917)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:49.961184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:50.021432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a2f245a8aac4b76a8fbb2eba1386917', 'request_id': 'req-04731415-2ced-40d7-a750-d72ee719356f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.027449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a2f245a8aac4b76a8fbb2eba1386917', 'request_id': 'req-04731415-2ced-40d7-a750-d72ee719356f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.035868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a2f245a8aac4b76a8fbb2eba1386917', 'request_id': 'req-04731415-2ced-40d7-a750-d72ee719356f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.048389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] The token cache is being invalidate because project 7a2f245a8aac4b76a8fbb2eba1386917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:50.049634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.050131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.051686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1040/2102] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:49 2022] DELETE /identity/v3/projects/7a2f245a8aac4b76a8fbb2eba1386917 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:50.055841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:50.066136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edade50-127e-4de1-a799-adddce2c8ceb service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:50.126714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:50.127861 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:50.128133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:50.128400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] PATH_INFO: `/v3/users/5fe9e621590b465188168929afc59bac` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:50.129533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe9e621590b465188168929afc59bac)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:50.131483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:50.181094 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe9e621590b465188168929afc59bac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.188101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe9e621590b465188168929afc59bac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.202550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe9e621590b465188168929afc59bac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.213797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe9e621590b465188168929afc59bac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.214428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9910fa84-a0d4-4dea-8469-d6a91b3f4546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe9e621590b465188168929afc59bac'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.222520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1063/2103] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:13:50 2022] DELETE /identity/v3/users/5fe9e621590b465188168929afc59bac => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:50.554203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:50.555734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:50.570134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04731415-2ced-40d7-a750-d72ee719356f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:50.584028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:50.631908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:50.632482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:50.632809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:50.633105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] PATH_INFO: `/v3/projects/97209e4ecab24cd9ae352bd2f2b76ff8` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:50.634345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97209e4ecab24cd9ae352bd2f2b76ff8)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:50.636293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:50.684729 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97209e4ecab24cd9ae352bd2f2b76ff8', 'request_id': 'req-20ff745e-0763-4225-bc3e-7245ecfea171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.690901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97209e4ecab24cd9ae352bd2f2b76ff8', 'request_id': 'req-20ff745e-0763-4225-bc3e-7245ecfea171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.698757 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97209e4ecab24cd9ae352bd2f2b76ff8', 'request_id': 'req-20ff745e-0763-4225-bc3e-7245ecfea171'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.710714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] The token cache is being invalidate because project 97209e4ecab24cd9ae352bd2f2b76ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:13:50.711000 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.711623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-20ff745e-0763-4225-bc3e-7245ecfea171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:13:50.713086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1041/2104] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:13:50 2022] DELETE /identity/v3/projects/97209e4ecab24cd9ae352bd2f2b76ff8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:50.718254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12a4cc-88f7-41d3-b10e-5a5d24845d6d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:50.718716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12a4cc-88f7-41d3-b10e-5a5d24845d6d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:50.718825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12a4cc-88f7-41d3-b10e-5a5d24845d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:50.739999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1b12a4cc-88f7-41d3-b10e-5a5d24845d6d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:50.741277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b12a4cc-88f7-41d3-b10e-5a5d24845d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:50.786337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:50.810378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1064/2105] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:50.818554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d79f1-1a4c-4175-8d9e-bf3aaa3bcf73 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:50.818837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d79f1-1a4c-4175-8d9e-bf3aaa3bcf73 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:50.819114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d79f1-1a4c-4175-8d9e-bf3aaa3bcf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:50.839942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ac7d79f1-1a4c-4175-8d9e-bf3aaa3bcf73 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:50.841291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ac7d79f1-1a4c-4175-8d9e-bf3aaa3bcf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:50.850868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1042/2106] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:50.855124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12a4cc-88f7-41d3-b10e-5a5d24845d6d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:50.865089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12a4cc-88f7-41d3-b10e-5a5d24845d6d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:50.869922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:50.870555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:50.870834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:50.871103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:50.871746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:13:50.872047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:50.873654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:50.886752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1065/2107] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:13:50.890801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d79f1-1a4c-4175-8d9e-bf3aaa3bcf73 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:50.901656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d79f1-1a4c-4175-8d9e-bf3aaa3bcf73 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:50.906588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:50.907240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:50.907520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:50.907811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:50.908571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:50.910519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:50.939034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1043/2108] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:50 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:50.943988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:50.954146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a995e-18ea-422f-a32c-6b557598baee admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.011516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.012126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.012406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.012732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.013366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.015315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.039092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1066/2109] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:50 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:13:51.042901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.054536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06b4ef-e9f3-4f99-9f94-158f1f7f2a67 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.059914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.060628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.060985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.061337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.062105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.063775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.078223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1044/2110] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:51.079604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.091210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3052e1-195d-4ac0-847d-d8b866513457 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.095954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae88c83-a51f-4ba5-b5fe-57ca12e44c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.096602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88c83-a51f-4ba5-b5fe-57ca12e44c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.096899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88c83-a51f-4ba5-b5fe-57ca12e44c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.097169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88c83-a51f-4ba5-b5fe-57ca12e44c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.097999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae88c83-a51f-4ba5-b5fe-57ca12e44c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.099655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae88c83-a51f-4ba5-b5fe-57ca12e44c73 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.113817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1067/2111] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:51.118296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.130648 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76025449-a807-4af1-8d0e-cd0a6d52596b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.135872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.136666 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.137698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.137698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] PATH_INFO: `/v3/projects/1431b07cc711495c9f862e4dabde7b9e/users/3d6142d1cb1741bfb15f010e3e64a02c/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.163655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1431b07cc711495c9f862e4dabde7b9e, user_id=3d6142d1cb1741bfb15f010e3e64a02c, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.166482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.180771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1045/2112] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:51 2022] PUT /identity/v3/projects/1431b07cc711495c9f862e4dabde7b9e/users/3d6142d1cb1741bfb15f010e3e64a02c/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:13:51.192254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218739c-792a-45d3-9d0e-b9ef2df1660a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.192604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218739c-792a-45d3-9d0e-b9ef2df1660a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.192963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218739c-792a-45d3-9d0e-b9ef2df1660a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.221566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6218739c-792a-45d3-9d0e-b9ef2df1660a None None] MFA Rules not processed for user `3d6142d1cb1741bfb15f010e3e64a02c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:51.222827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6218739c-792a-45d3-9d0e-b9ef2df1660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:51.296607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1068/2113] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:13:51.297808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.358212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:51.391057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c75e9e-c554-4630-af6d-1750ad50ca43 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.396262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.396880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.397159 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.397434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.398101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.400354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.441160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1046/2114] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:13:51 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:51.442635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218739c-792a-45d3-9d0e-b9ef2df1660a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.456158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218739c-792a-45d3-9d0e-b9ef2df1660a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.499849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:51.537084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.537588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.538188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.538526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.539272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.541885 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.573417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1069/2115] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:13:51 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:13:51.574652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.585840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50c79a-7395-4b33-a2c2-2001e512c7a6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.590469 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.591067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.591343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.591614 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.592253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.593852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.604853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1047/2116] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:13:51.609019 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.619541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a506f-7d79-4f24-af4d-0aa3322586b4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.625457 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.626161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.626442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.626738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] PATH_INFO: `/v3/projects/4cec0af3800b4ae3846eb2cd5de40c53/users/9c2f3d7f51cf4dbb98a3bd9f0b6f49fd/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.642787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cec0af3800b4ae3846eb2cd5de40c53, user_id=9c2f3d7f51cf4dbb98a3bd9f0b6f49fd, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.645481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.659947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1070/2117] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:51 2022] PUT /identity/v3/projects/4cec0af3800b4ae3846eb2cd5de40c53/users/9c2f3d7f51cf4dbb98a3bd9f0b6f49fd/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:51.664961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.675719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c782ee-9f2c-42ac-ab09-a58c897b8f78 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.732853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2d85f-f864-4b4c-8904-b1e2c2cf8a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.733455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d85f-f864-4b4c-8904-b1e2c2cf8a65 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.733750 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d85f-f864-4b4c-8904-b1e2c2cf8a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.734032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2d85f-f864-4b4c-8904-b1e2c2cf8a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.734692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2d85f-f864-4b4c-8904-b1e2c2cf8a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.737145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2d85f-f864-4b4c-8904-b1e2c2cf8a65 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.747081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1048/2118] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:13:51.751379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.761932 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8315a-bea2-461c-801d-f6479402afb4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.766949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.767621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.767895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.768165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] PATH_INFO: `/v3/projects/4cec0af3800b4ae3846eb2cd5de40c53/users/9c2f3d7f51cf4dbb98a3bd9f0b6f49fd/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.770141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cec0af3800b4ae3846eb2cd5de40c53, user_id=9c2f3d7f51cf4dbb98a3bd9f0b6f49fd, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.773013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.784042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1071/2119] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:13:51 2022] PUT /identity/v3/projects/4cec0af3800b4ae3846eb2cd5de40c53/users/9c2f3d7f51cf4dbb98a3bd9f0b6f49fd/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:13:51.794456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94d1a7-0cac-44e0-95ae-2c152943d289 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.794746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94d1a7-0cac-44e0-95ae-2c152943d289 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.795023 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94d1a7-0cac-44e0-95ae-2c152943d289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.825399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-0b94d1a7-0cac-44e0-95ae-2c152943d289 None None] MFA Rules not processed for user `9c2f3d7f51cf4dbb98a3bd9f0b6f49fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:13:51.826660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b94d1a7-0cac-44e0-95ae-2c152943d289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:51.899228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1049/2120] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:13:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:13:51.910883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:51.912670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:13:51.971769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f0ed1-3dca-4bda-ad01-d5485efa55d2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:51.975732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:51.976345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:51.976623 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:51.976895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:51.987164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:51.988595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:51.997975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1072/2121] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:13:52.048589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94d1a7-0cac-44e0-95ae-2c152943d289 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:13:52.058452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94d1a7-0cac-44e0-95ae-2c152943d289 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:13:52.062298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d4f99-6d59-4da7-9b0f-44ef30a0e9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:13:52.063024 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d4f99-6d59-4da7-9b0f-44ef30a0e9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:13:52.063305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d4f99-6d59-4da7-9b0f-44ef30a0e9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:13:52.063608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d4f99-6d59-4da7-9b0f-44ef30a0e9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:13:52.104132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:13:52.128533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d4f99-6d59-4da7-9b0f-44ef30a0e9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:13:52.130371 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d4f99-6d59-4da7-9b0f-44ef30a0e9e6 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:13:52.140037 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1050/2122] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:13:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:00.778594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:00.855251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e646f4b-3fc1-4176-a381-2b861901bc81 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:00.859912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:00.860544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:00.860824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:00.861093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] PATH_INFO: `/v3/users/3d6142d1cb1741bfb15f010e3e64a02c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:00.862304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d6142d1cb1741bfb15f010e3e64a02c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:00.864241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:00.914470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:00.928364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d6142d1cb1741bfb15f010e3e64a02c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:00.937972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d6142d1cb1741bfb15f010e3e64a02c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:00.955241 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d6142d1cb1741bfb15f010e3e64a02c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:00.964334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d6142d1cb1741bfb15f010e3e64a02c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:00.965229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d6142d1cb1741bfb15f010e3e64a02c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:00.972821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1073/2123] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:00 2022] DELETE /identity/v3/users/3d6142d1cb1741bfb15f010e3e64a02c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:14:00.980868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:00.980868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:00.980868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:00.997872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:00.999072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:01.063188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1051/2124] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:01.073453 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:01.074911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:01.133403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e996a-d56f-4e03-8d2c-76a6ab3a634e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:01.137537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:01.138177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:01.138454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:01.138723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:01.148804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:01.150267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:01.159911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1074/2125] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:01.459473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:01.461914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:01.483201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb22ea5-3c22-4cdd-965a-ad79e3ba7c30 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:01.495732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:01.496389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:01.496672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:01.496949 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] PATH_INFO: `/v3/projects/1431b07cc711495c9f862e4dabde7b9e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:01.498811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1431b07cc711495c9f862e4dabde7b9e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:01.500742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:01.552310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1431b07cc711495c9f862e4dabde7b9e', 'request_id': 'req-3106a9b4-7880-4ecf-8c45-a00fe992a384'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.557836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1431b07cc711495c9f862e4dabde7b9e', 'request_id': 'req-3106a9b4-7880-4ecf-8c45-a00fe992a384'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.568805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1431b07cc711495c9f862e4dabde7b9e', 'request_id': 'req-3106a9b4-7880-4ecf-8c45-a00fe992a384'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.580213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] The token cache is being invalidate because project 1431b07cc711495c9f862e4dabde7b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:01.580647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.581543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.583114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1052/2126] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:01 2022] DELETE /identity/v3/projects/1431b07cc711495c9f862e4dabde7b9e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:01.587031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:01.598544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cd76e-f8ec-4e2c-9d6a-ccf3cd62c8f2 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:01.654777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:01.655394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:01.655672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:01.655944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] PATH_INFO: `/v3/users/9c2f3d7f51cf4dbb98a3bd9f0b6f49fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:01.657052 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c2f3d7f51cf4dbb98a3bd9f0b6f49fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:01.659583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:01.711170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c2f3d7f51cf4dbb98a3bd9f0b6f49fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.718374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c2f3d7f51cf4dbb98a3bd9f0b6f49fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.719479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:01.734826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c2f3d7f51cf4dbb98a3bd9f0b6f49fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.744190 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c2f3d7f51cf4dbb98a3bd9f0b6f49fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.744940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc876fd8-c541-42b8-8c59-7886a0bbec64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c2f3d7f51cf4dbb98a3bd9f0b6f49fd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:01.752056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1075/2127] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:01 2022] DELETE /identity/v3/users/9c2f3d7f51cf4dbb98a3bd9f0b6f49fd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:02.100525 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.103845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:02.122155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106a9b4-7880-4ecf-8c45-a00fe992a384 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.207790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.208561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.208942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.209335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] PATH_INFO: `/v3/projects/4cec0af3800b4ae3846eb2cd5de40c53` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.210685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cec0af3800b4ae3846eb2cd5de40c53)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.212797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.215603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:02.271641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cec0af3800b4ae3846eb2cd5de40c53', 'request_id': 'req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:02.278026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cec0af3800b4ae3846eb2cd5de40c53', 'request_id': 'req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:02.286642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cec0af3800b4ae3846eb2cd5de40c53', 'request_id': 'req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:02.298887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] The token cache is being invalidate because project 4cec0af3800b4ae3846eb2cd5de40c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:02.299927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:02.300759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b0555a5-4b74-49ee-aeff-a6424f9a9e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:02.302346 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1053/2128] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:02 2022] DELETE /identity/v3/projects/4cec0af3800b4ae3846eb2cd5de40c53 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:02.307633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5eda0-4f25-48b5-b051-000b3cd51b9b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.308227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5eda0-4f25-48b5-b051-000b3cd51b9b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.308227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5eda0-4f25-48b5-b051-000b3cd51b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.327818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-05f5eda0-4f25-48b5-b051-000b3cd51b9b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:02.329041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05f5eda0-4f25-48b5-b051-000b3cd51b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:02.406038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1076/2129] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:14:02.414450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f5be2-e6f2-4ab1-a0f6-9207ac593990 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.414734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f5be2-e6f2-4ab1-a0f6-9207ac593990 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.415003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f5be2-e6f2-4ab1-a0f6-9207ac593990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.436554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1f8f5be2-e6f2-4ab1-a0f6-9207ac593990 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:02.437818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1f8f5be2-e6f2-4ab1-a0f6-9207ac593990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:02.446381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1054/2130] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:02.450832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5eda0-4f25-48b5-b051-000b3cd51b9b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.461043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5eda0-4f25-48b5-b051-000b3cd51b9b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.466293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.466889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.467170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.467443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.468143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:14:02.468460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.470158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.482436 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1077/2131] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:14:02.488432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f5be2-e6f2-4ab1-a0f6-9207ac593990 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.497825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f5be2-e6f2-4ab1-a0f6-9207ac593990 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.502739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.503447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.503781 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.504145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.504794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.506810 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.539919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1055/2132] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:14:02 2022] POST /identity/v3/projects => generated 407 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:14:02.544278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.554505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f6cb5-1337-4252-92fe-b5860985da7f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.615911 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.616517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.616798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.617072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.617945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.619993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.645080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1078/2133] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:14:02 2022] POST /identity/v3/users => generated 304 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:02.649381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.660152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d492-15e8-4720-b8de-0d130d032150 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.668278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.668892 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.669353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.669629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.670326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.672249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.683321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1056/2134] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:14:02.687350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.698450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0167-27b8-4931-8f02-46ba9369ce24 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.703125 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9c5c8-8d9b-424f-a245-2d99020a7c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.703709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c5c8-8d9b-424f-a245-2d99020a7c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.703983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c5c8-8d9b-424f-a245-2d99020a7c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.704252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9c5c8-8d9b-424f-a245-2d99020a7c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.704880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9c5c8-8d9b-424f-a245-2d99020a7c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.707239 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9c5c8-8d9b-424f-a245-2d99020a7c03 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.717965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1079/2135] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:02.722985 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.733235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad46ac8-314c-4dd1-91c7-4b40a4f20a5f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.738264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.739041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.739343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.739615 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] PATH_INFO: `/v3/projects/64f397f56e7f4e6985e667c52a7cc9fd/users/a6cda7ab05804dfe9d057e05e1b93d76/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.753875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f397f56e7f4e6985e667c52a7cc9fd, user_id=a6cda7ab05804dfe9d057e05e1b93d76, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.756464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.768968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1057/2136] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:02 2022] PUT /identity/v3/projects/64f397f56e7f4e6985e667c52a7cc9fd/users/a6cda7ab05804dfe9d057e05e1b93d76/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:02.781033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d5c5a-8a3c-4cae-9c3e-4f41fda9be86 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.781353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d5c5a-8a3c-4cae-9c3e-4f41fda9be86 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.781632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d5c5a-8a3c-4cae-9c3e-4f41fda9be86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.810164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-493d5c5a-8a3c-4cae-9c3e-4f41fda9be86 None None] MFA Rules not processed for user `a6cda7ab05804dfe9d057e05e1b93d76`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:02.811410 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-493d5c5a-8a3c-4cae-9c3e-4f41fda9be86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:02.841817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:02.880910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1080/2137] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:02 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:02.885419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.946696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745827d3-9681-44c0-9d6e-f9e7d21b8bcc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:02.951348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:02.951939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:02.952215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:02.952488 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:02.953120 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:02.955019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:02.982107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1058/2138] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:14:02 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:02.986113 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d5c5a-8a3c-4cae-9c3e-4f41fda9be86 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:02.996854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d5c5a-8a3c-4cae-9c3e-4f41fda9be86 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:03.052238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:03.052838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.053119 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.053390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.054051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:03.055959 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:03.081070 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1081/2139] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:14:02 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:03.082975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:03.093109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685e6d-f01e-467a-a54b-d6a0c77cb914 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:03.097769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:03.098362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.098642 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.098914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.099636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:03.101239 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:03.111624 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1059/2140] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:14:03.115773 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:03.125671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a5da5-9b13-494f-8485-486920e73470 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:03.130979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:03.131567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.131846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.132121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] PATH_INFO: `/v3/projects/589889b086ed463285d2d084a52c7a92/users/ecc5af1a2f834e49bcb989b36688e52c/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.147513 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=589889b086ed463285d2d084a52c7a92, user_id=ecc5af1a2f834e49bcb989b36688e52c, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:03.150253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:03.162709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1082/2141] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:03 2022] PUT /identity/v3/projects/589889b086ed463285d2d084a52c7a92/users/ecc5af1a2f834e49bcb989b36688e52c/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:03.166532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:03.171681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:03.178247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305945-048e-4a13-827d-29977d91ad65 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:03.241325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4e119-4864-4c0a-8e9a-c1b9dee0344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:03.241960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4e119-4864-4c0a-8e9a-c1b9dee0344f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.242237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4e119-4864-4c0a-8e9a-c1b9dee0344f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.242509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4e119-4864-4c0a-8e9a-c1b9dee0344f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.243172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4e119-4864-4c0a-8e9a-c1b9dee0344f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:03.244771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4e119-4864-4c0a-8e9a-c1b9dee0344f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:03.255416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1060/2142] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:03.259629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:03.270203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9751561-1a08-49e4-9043-beeac5c77a3f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:03.275219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:03.275938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.276286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.276847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] PATH_INFO: `/v3/projects/589889b086ed463285d2d084a52c7a92/users/ecc5af1a2f834e49bcb989b36688e52c/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.278766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=589889b086ed463285d2d084a52c7a92, user_id=ecc5af1a2f834e49bcb989b36688e52c, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:03.281401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:03.292883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1083/2143] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:03 2022] PUT /identity/v3/projects/589889b086ed463285d2d084a52c7a92/users/ecc5af1a2f834e49bcb989b36688e52c/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:03.303699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106774-0748-4c13-a46d-aa00bbeb69d5 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.303987 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106774-0748-4c13-a46d-aa00bbeb69d5 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.304313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106774-0748-4c13-a46d-aa00bbeb69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.332746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-d8106774-0748-4c13-a46d-aa00bbeb69d5 None None] MFA Rules not processed for user `ecc5af1a2f834e49bcb989b36688e52c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:03.333992 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8106774-0748-4c13-a46d-aa00bbeb69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:03.413260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1061/2144] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:03 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:03.458378 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:03.460217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:03.535062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ecec5-76e1-4259-acdb-fe54f575dd23 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:03.539181 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:03.539783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.540076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.540375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.551858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:03.553299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:03.569829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1084/2145] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:03 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:03.788143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8106774-0748-4c13-a46d-aa00bbeb69d5 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:03.801207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8106774-0748-4c13-a46d-aa00bbeb69d5 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:03.806013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d7973-5548-47ae-8a22-23af1e91242a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:03.806767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7973-5548-47ae-8a22-23af1e91242a service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:03.807165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7973-5548-47ae-8a22-23af1e91242a service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:03.807552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d7973-5548-47ae-8a22-23af1e91242a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:03.936306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d7973-5548-47ae-8a22-23af1e91242a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:03.938674 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d7973-5548-47ae-8a22-23af1e91242a service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:03.950070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1062/2146] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:03 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:06.205480 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:06.207035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:06.238872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:06.279696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997f21b-97a8-42dc-b039-bfed318f4127 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:06.284845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:06.285467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:06.285767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:06.286069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] PATH_INFO: `/v3/users/31bf07c8b8414a66be9dbe0f0035d364` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:06.287191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31bf07c8b8414a66be9dbe0f0035d364)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:06.289174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:06.337420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31bf07c8b8414a66be9dbe0f0035d364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.344400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31bf07c8b8414a66be9dbe0f0035d364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.356851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31bf07c8b8414a66be9dbe0f0035d364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.366570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31bf07c8b8414a66be9dbe0f0035d364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.367322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31bf07c8b8414a66be9dbe0f0035d364'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.376576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1085/2147] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:06 2022] DELETE /identity/v3/users/31bf07c8b8414a66be9dbe0f0035d364 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:06.379394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:06.379881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:06.379969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:06.401881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:06.403063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:06.420302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:06.470344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1063/2148] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:06.481633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:06.483355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:06.547650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c667c-edae-4df6-8710-000baf1087d1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:06.551630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596ed4-faa4-476a-a249-a275d43a932d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:06.552243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596ed4-faa4-476a-a249-a275d43a932d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:06.552526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596ed4-faa4-476a-a249-a275d43a932d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:06.552795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596ed4-faa4-476a-a249-a275d43a932d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:06.562529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92596ed4-faa4-476a-a249-a275d43a932d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:06.563950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92596ed4-faa4-476a-a249-a275d43a932d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:06.573598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1086/2149] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:06.837722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:06.839182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:06.852149 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fb96f-be13-48a5-8873-8b2ba7cfe0ec None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:06.857387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:06.858144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:06.858524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:06.858903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] PATH_INFO: `/v3/projects/674ceeddd4ed4a0fbc181bfeea2a0f30` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:06.860180 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=674ceeddd4ed4a0fbc181bfeea2a0f30)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:06.862211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:06.920210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '674ceeddd4ed4a0fbc181bfeea2a0f30', 'request_id': 'req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.925885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '674ceeddd4ed4a0fbc181bfeea2a0f30', 'request_id': 'req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.933671 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '674ceeddd4ed4a0fbc181bfeea2a0f30', 'request_id': 'req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.945140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] The token cache is being invalidate because project 674ceeddd4ed4a0fbc181bfeea2a0f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:06.945428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.946053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-b27b4fe0-2bd4-464a-ab9b-e626bde9bc50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:06.947523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1064/2150] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:06 2022] DELETE /identity/v3/projects/674ceeddd4ed4a0fbc181bfeea2a0f30 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:06.952383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da75c3-ea87-48e1-b741-36e2aa301358 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:06.952663 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da75c3-ea87-48e1-b741-36e2aa301358 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:06.952929 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da75c3-ea87-48e1-b741-36e2aa301358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:06.973061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-19da75c3-ea87-48e1-b741-36e2aa301358 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:06.974386 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19da75c3-ea87-48e1-b741-36e2aa301358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:07.019395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:07.037237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1087/2151] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:07.045140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076c019-c412-4428-845a-af77080b9fe6 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.045429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076c019-c412-4428-845a-af77080b9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.045700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076c019-c412-4428-845a-af77080b9fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.067399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-a076c019-c412-4428-845a-af77080b9fe6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:07.068636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a076c019-c412-4428-845a-af77080b9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:07.076738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1065/2152] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:07.081438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da75c3-ea87-48e1-b741-36e2aa301358 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:07.091435 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da75c3-ea87-48e1-b741-36e2aa301358 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:07.096500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:07.097202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.097401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.097690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.098363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:14:07.098672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:07.100305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:07.116317 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1088/2153] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:14:07.117395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076c019-c412-4428-845a-af77080b9fe6 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:07.126771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076c019-c412-4428-845a-af77080b9fe6 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:07.132353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:07.133056 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.133345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.133619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.134294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:07.136193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:07.168382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1066/2154] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:14:07 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:07.172603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:07.183271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632426f-e16a-4dfc-ba78-d6b5532561aa admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:07.247502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:07.248149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.248467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.248737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.249415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:07.251501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:07.278304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1089/2155] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:14:07 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:07.279839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:07.291165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478a2fd-5902-4e36-91fd-a2338a7864df admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:07.295989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:07.296665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.297017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.297364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.298116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:07.299867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:07.312807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1067/2156] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:07.317136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:07.327209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e0da8-6df0-4054-beed-844af4dd77e9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:07.332864 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efeee1d-ee12-4b9e-b40b-a6237e36a900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:07.333405 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeee1d-ee12-4b9e-b40b-a6237e36a900 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.333683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeee1d-ee12-4b9e-b40b-a6237e36a900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.333989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeee1d-ee12-4b9e-b40b-a6237e36a900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.334629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efeee1d-ee12-4b9e-b40b-a6237e36a900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:07.336231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efeee1d-ee12-4b9e-b40b-a6237e36a900 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:07.349077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1090/2157] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:07.353367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:07.363948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b77094-5964-464f-bda8-d7703f5740f8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:07.368964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:07.369780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.370222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.370566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] PATH_INFO: `/v3/projects/e9946e78bebe4d2a8d32a39120b0b978/users/b894ea45ef5f4dcca7ced8cd7834a25a/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.388767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9946e78bebe4d2a8d32a39120b0b978, user_id=b894ea45ef5f4dcca7ced8cd7834a25a, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:07.391550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:07.406691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1068/2158] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:07 2022] PUT /identity/v3/projects/e9946e78bebe4d2a8d32a39120b0b978/users/b894ea45ef5f4dcca7ced8cd7834a25a/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:07.411733 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2fb4e-207f-439d-aa45-267cd82ac182 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.412018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2fb4e-207f-439d-aa45-267cd82ac182 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.412365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2fb4e-207f-439d-aa45-267cd82ac182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.435233 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-47e2fb4e-207f-439d-aa45-267cd82ac182 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:07.436474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47e2fb4e-207f-439d-aa45-267cd82ac182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:07.497630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1091/2159] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:07.510231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:07.511947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:07.558610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:07.577350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbce1a-fa37-41e0-a60b-87284638e76c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:07.577350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:07.577350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:07.577715 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:07.578306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:07.590738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:07.592142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:07.602609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1069/2160] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:12.127229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4daf01-e8b0-4840-9d2a-9f8366c7fc1e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:12.128028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4daf01-e8b0-4840-9d2a-9f8366c7fc1e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:12.128028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4daf01-e8b0-4840-9d2a-9f8366c7fc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:12.167499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ec4daf01-e8b0-4840-9d2a-9f8366c7fc1e None None] MFA Rules not processed for user `b894ea45ef5f4dcca7ced8cd7834a25a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:12.168665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec4daf01-e8b0-4840-9d2a-9f8366c7fc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:12.218972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:12.249944 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1092/2161] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:14:12.257154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:12.271244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bd022-6e0a-4df7-abcd-8dac07e86450 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:12.277025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:12.277662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:12.277991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:12.278296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:12.279213 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:12.281129 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:12.323182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4daf01-e8b0-4840-9d2a-9f8366c7fc1e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:12.324482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1070/2162] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:14:12 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:12.343922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4daf01-e8b0-4840-9d2a-9f8366c7fc1e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:12.431965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:12.432844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:12.433218 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:12.434303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:12.435163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:12.437539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:12.479027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1093/2163] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:14:12 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:12.483089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:12.495252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927cdf2-f9e4-4a3f-b0ff-a36c74343ca9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:12.501182 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:12.501914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:12.501914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:12.502657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:12.504022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:12.505558 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:12.518852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1071/2164] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:12.524304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:12.536754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d2c2-1cc7-4fbb-95fa-928b134c5d42 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:12.542506 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:12.543255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:12.543564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:12.543867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] PATH_INFO: `/v3/projects/296437204fd44dc9b81cd68f33f279f6/users/f68fc0b8576b46dfa93566082342063e/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:12.565235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=296437204fd44dc9b81cd68f33f279f6, user_id=f68fc0b8576b46dfa93566082342063e, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:12.568158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:12.587005 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1094/2165] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:12 2022] PUT /identity/v3/projects/296437204fd44dc9b81cd68f33f279f6/users/f68fc0b8576b46dfa93566082342063e/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:12.591822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:12.604473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf12e7-28bb-4faa-9d4d-76bebf82dbf8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:12.675253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:12.675941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:12.676168 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:12.676426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:12.677081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:12.679035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:12.695160 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1072/2166] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:12.701459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:12.714174 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f11a56-58c7-47ec-bcbb-55999eda4819 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:12.719333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c1775-e6b6-455d-b3cf-0a109c4bc81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:12.720043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1775-e6b6-455d-b3cf-0a109c4bc81c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:12.720322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1775-e6b6-455d-b3cf-0a109c4bc81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:12.720599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1775-e6b6-455d-b3cf-0a109c4bc81c admin admin] PATH_INFO: `/v3/projects/296437204fd44dc9b81cd68f33f279f6/users/f68fc0b8576b46dfa93566082342063e/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:12.723177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c1775-e6b6-455d-b3cf-0a109c4bc81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=296437204fd44dc9b81cd68f33f279f6, user_id=f68fc0b8576b46dfa93566082342063e, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:12.726169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c1775-e6b6-455d-b3cf-0a109c4bc81c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:12.743384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1095/2167] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:12 2022] PUT /identity/v3/projects/296437204fd44dc9b81cd68f33f279f6/users/f68fc0b8576b46dfa93566082342063e/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:16.041522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:16.043525 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:16.070331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335d1b-bda2-4f1e-b3cd-b8e20be00b14 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:16.229926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:16.229926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:16.229926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:16.229926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] PATH_INFO: `/v3/users/a6cda7ab05804dfe9d057e05e1b93d76` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:16.231275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6cda7ab05804dfe9d057e05e1b93d76)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:16.233335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:16.385318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6cda7ab05804dfe9d057e05e1b93d76'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:16.389439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:16.404240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6cda7ab05804dfe9d057e05e1b93d76'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:16.447319 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6cda7ab05804dfe9d057e05e1b93d76'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:16.453267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6cda7ab05804dfe9d057e05e1b93d76'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:16.453267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6cda7ab05804dfe9d057e05e1b93d76'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:16.502020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1073/2168] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:16 2022] DELETE /identity/v3/users/a6cda7ab05804dfe9d057e05e1b93d76 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:16.506765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:16.507301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:16.507622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:16.560690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:16.563068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:16.699593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1096/2169] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:16.712428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:16.714273 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:16.790181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe625-6646-4880-98a0-9aa8658cd71a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:16.795080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:16.795863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:16.796247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:16.796723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:16.809850 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:16.811867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:16.824751 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1074/2170] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:14:17.540572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:17.544390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:17.570411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195f23-f888-48cc-9191-b0da83debeee None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:17.582975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:17.583999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:17.584288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:17.584557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] PATH_INFO: `/v3/projects/64f397f56e7f4e6985e667c52a7cc9fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:17.591782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f397f56e7f4e6985e667c52a7cc9fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:17.598128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:17.675991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:17.699217 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f397f56e7f4e6985e667c52a7cc9fd', 'request_id': 'req-3f85d61e-edfc-443e-bcf5-4101a5f25e71'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.706852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f397f56e7f4e6985e667c52a7cc9fd', 'request_id': 'req-3f85d61e-edfc-443e-bcf5-4101a5f25e71'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.725385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f397f56e7f4e6985e667c52a7cc9fd', 'request_id': 'req-3f85d61e-edfc-443e-bcf5-4101a5f25e71'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.747605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] The token cache is being invalidate because project 64f397f56e7f4e6985e667c52a7cc9fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:17.747956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.748629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.753550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1097/2171] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:17 2022] DELETE /identity/v3/projects/64f397f56e7f4e6985e667c52a7cc9fd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:17.755125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:17.774619 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc4985-3203-4005-bba4-44d3270ad180 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:17.869972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:17.870922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:17.871395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:17.871839 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] PATH_INFO: `/v3/users/ecc5af1a2f834e49bcb989b36688e52c` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:17.873388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc5af1a2f834e49bcb989b36688e52c)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:17.876038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:17.949944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc5af1a2f834e49bcb989b36688e52c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.957977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc5af1a2f834e49bcb989b36688e52c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.974206 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:17.975290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc5af1a2f834e49bcb989b36688e52c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.976408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc5af1a2f834e49bcb989b36688e52c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:17.977682 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-17c7b561-22e8-499c-9563-d7f5d5a65ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc5af1a2f834e49bcb989b36688e52c'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:18.002169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1075/2172] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:17 2022] DELETE /identity/v3/users/ecc5af1a2f834e49bcb989b36688e52c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:18.367590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:18.370142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:18.396779 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85d61e-edfc-443e-bcf5-4101a5f25e71 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:18.482112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:18.482402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:18.482686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:18.510637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:18.511716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:18.512066 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:18.512866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] PATH_INFO: `/v3/projects/589889b086ed463285d2d084a52c7a92` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:18.514576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=589889b086ed463285d2d084a52c7a92)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:18.515317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] MFA Rules not processed for user `f68fc0b8576b46dfa93566082342063e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:18.516606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:18.517165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:18.592807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589889b086ed463285d2d084a52c7a92', 'request_id': 'req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:18.602705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589889b086ed463285d2d084a52c7a92', 'request_id': 'req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:18.609612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589889b086ed463285d2d084a52c7a92', 'request_id': 'req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:18.618408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1076/2173] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:18.629752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] The token cache is being invalidate because project 589889b086ed463285d2d084a52c7a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:18.630166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:18.631483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:18.632799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8a6b3e38-dacf-43ef-9a27-a08ad2517f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:18.633381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:18.634888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1098/2174] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:18 2022] DELETE /identity/v3/projects/589889b086ed463285d2d084a52c7a92 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:18.640888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487590d-d7a0-41f3-88cf-cc89af4f9a5d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:18.641486 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487590d-d7a0-41f3-88cf-cc89af4f9a5d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:18.641615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487590d-d7a0-41f3-88cf-cc89af4f9a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:18.670517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8487590d-d7a0-41f3-88cf-cc89af4f9a5d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:18.671008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8487590d-d7a0-41f3-88cf-cc89af4f9a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:18.711710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d04ff-be29-4ad7-8307-a23a2e4bd05d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:18.734693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:18.762299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1099/2175] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:18.771176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3893cd-ae5d-4cbd-8ede-7a2cf08a25c6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:18.771538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3893cd-ae5d-4cbd-8ede-7a2cf08a25c6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:18.771885 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3893cd-ae5d-4cbd-8ede-7a2cf08a25c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:18.774036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7214e6addc646a899efb82283ededc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:18.774717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:18.775073 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:18.775390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:18.777176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:18.793993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9e3893cd-ae5d-4cbd-8ede-7a2cf08a25c6 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:18.797845 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e3893cd-ae5d-4cbd-8ede-7a2cf08a25c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:18.809593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1100/2176] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:14:18.813308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3893cd-ae5d-4cbd-8ede-7a2cf08a25c6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:18.823565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3893cd-ae5d-4cbd-8ede-7a2cf08a25c6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:18.828333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:18.828995 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:18.829386 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:18.829705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:18.831029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:14:18.831397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:18.833099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:18.833418 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:18.847677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1101/2177] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:14:18.851065 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:18.852919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:18.854145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:18.863453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1077/2178] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:14:18.866022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83528d66-10ec-431b-8aa4-3fc7c27046ed admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:18.871097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550dcc0-52cc-45c1-b860-f1fb856f8f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:18.871851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550dcc0-52cc-45c1-b860-f1fb856f8f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:18.872451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550dcc0-52cc-45c1-b860-f1fb856f8f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:18.872728 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550dcc0-52cc-45c1-b860-f1fb856f8f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:18.877222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550dcc0-52cc-45c1-b860-f1fb856f8f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:18.877222 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550dcc0-52cc-45c1-b860-f1fb856f8f71 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:18.910381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1102/2179] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:14:18 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:18.914618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:18.927496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb80d71-94ac-4cfb-8a6a-40dbe1abf595 service glance] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:19.042432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:19.043453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.044326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.045334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.046203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:19.051486 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:19.064053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c3f4-a173-4eb6-9691-c20987874d3a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.064392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c3f4-a173-4eb6-9691-c20987874d3a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.064669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c3f4-a173-4eb6-9691-c20987874d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.094413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-70b9c3f4-a173-4eb6-9691-c20987874d3a None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:19.094939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1078/2180] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:14:18 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:14:19.097053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70b9c3f4-a173-4eb6-9691-c20987874d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:19.102274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:19.117111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e851f-b67c-49e0-82ac-ab1642380597 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:19.122854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:19.123478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.123756 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.124029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.124681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:19.127212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:19.146203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1079/2181] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:19.147216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:19.160298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7fb92-599f-491a-aca9-3aab53301c69 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:19.170611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:19.173945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.173945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.173945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.173945 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:19.177532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:19.185240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1103/2182] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:14:19.195089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1080/2183] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:19.199231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9c3f4-a173-4eb6-9691-c20987874d3a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:19.201139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:19.203241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:19.215751 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9c3f4-a173-4eb6-9691-c20987874d3a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:19.226203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eeda8-1dce-4358-a025-a13541c947af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:19.227604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eeda8-1dce-4358-a025-a13541c947af admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.227604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eeda8-1dce-4358-a025-a13541c947af admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.229001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eeda8-1dce-4358-a025-a13541c947af admin admin] PATH_INFO: `/v3/projects/fcef5c60d21344779c78c9c453e2c798/users/d1b7434c8a6b498cb37ca16ee23080ff/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.253609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eeda8-1dce-4358-a025-a13541c947af admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcef5c60d21344779c78c9c453e2c798, user_id=d1b7434c8a6b498cb37ca16ee23080ff, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:19.257046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eeda8-1dce-4358-a025-a13541c947af admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:19.289583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1104/2184] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:19 2022] PUT /identity/v3/projects/fcef5c60d21344779c78c9c453e2c798/users/d1b7434c8a6b498cb37ca16ee23080ff/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:19.292445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.292736 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.293004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.327809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079cda5-e032-4feb-abf7-48757efcef43 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:19.331982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] MFA Rules not processed for user `d1b7434c8a6b498cb37ca16ee23080ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:19.333220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:19.406780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:19.427893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1105/2185] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:19.445468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:19.456823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:19.460362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f574d9-c29c-43cb-a3c5-33f919bcdd46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:19.462117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f574d9-c29c-43cb-a3c5-33f919bcdd46 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.462402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f574d9-c29c-43cb-a3c5-33f919bcdd46 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.462681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f574d9-c29c-43cb-a3c5-33f919bcdd46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.541551 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:19.606379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8950a9-7c42-4855-883a-277d555682b5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:19.609694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f574d9-c29c-43cb-a3c5-33f919bcdd46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:19.610605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:19.611517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:19.611805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:19.612059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f574d9-c29c-43cb-a3c5-33f919bcdd46 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:19.612159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:19.621062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:19.622683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:19.630556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1081/2186] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:14:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:14:19.633646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1106/2187] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:14:29.618032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=29 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:29.628588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=18 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:30.710852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d8f2-e967-46e6-adac-0387a0bd12a9 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:30.712267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d8f2-e967-46e6-adac-0387a0bd12a9 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:30.713320 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d8f2-e967-46e6-adac-0387a0bd12a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:30.743672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6546d8f2-e967-46e6-adac-0387a0bd12a9 None None] MFA Rules not processed for user `b894ea45ef5f4dcca7ced8cd7834a25a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:30.744965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6546d8f2-e967-46e6-adac-0387a0bd12a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:30.843731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1082/2188] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:34.149354 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:34.150902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:34.216247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764eb4-4f54-4369-abba-c61ccacf906e service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:34.221067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:34.221692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:34.222012 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:34.222284 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] PATH_INFO: `/v3/users/e6c9ed60880745f9adab0e0c635674f9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:34.223449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c9ed60880745f9adab0e0c635674f9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:34.225396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:34.275677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c9ed60880745f9adab0e0c635674f9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:34.283114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c9ed60880745f9adab0e0c635674f9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:34.297075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c9ed60880745f9adab0e0c635674f9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:34.306345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c9ed60880745f9adab0e0c635674f9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:34.307712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c9ed60880745f9adab0e0c635674f9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:34.314878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1107/2189] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:34 2022] DELETE /identity/v3/users/e6c9ed60880745f9adab0e0c635674f9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:34.319521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:34.320025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:34.320137 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:34.340515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:34.341607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:34.408125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1083/2190] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:34.424277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:34.425682 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:34.489900 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871eb7b2-442c-4931-9b2c-dfafe10e90b1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:34.494044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a285b15-6b01-409b-8a2c-eeb3d607c3a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:34.494910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a285b15-6b01-409b-8a2c-eeb3d607c3a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:34.495184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a285b15-6b01-409b-8a2c-eeb3d607c3a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:34.495512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a285b15-6b01-409b-8a2c-eeb3d607c3a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:34.506979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a285b15-6b01-409b-8a2c-eeb3d607c3a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:34.508384 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a285b15-6b01-409b-8a2c-eeb3d607c3a9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:34.521063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1108/2191] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:34.954468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:34.955017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:34.977956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7f92e-f405-44d1-acb8-d0b232a0f0d2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:34.993205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:34.994022 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:34.994291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:34.994559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] PATH_INFO: `/v3/projects/e77dc690fa764e61b0a021fe46a99cbd` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:34.995783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77dc690fa764e61b0a021fe46a99cbd)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:34.997927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:35.062456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77dc690fa764e61b0a021fe46a99cbd', 'request_id': 'req-1a36f160-525b-4693-a8e3-356f8888f532'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:35.077460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77dc690fa764e61b0a021fe46a99cbd', 'request_id': 'req-1a36f160-525b-4693-a8e3-356f8888f532'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:35.106372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77dc690fa764e61b0a021fe46a99cbd', 'request_id': 'req-1a36f160-525b-4693-a8e3-356f8888f532'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:35.121440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] The token cache is being invalidate because project e77dc690fa764e61b0a021fe46a99cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:35.122204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:35.122950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:35.126698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1084/2192] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:34 2022] DELETE /identity/v3/projects/e77dc690fa764e61b0a021fe46a99cbd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:35.548517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452613a-7f1b-4250-b718-b783fde1e1a2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:35.549203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452613a-7f1b-4250-b718-b783fde1e1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:35.549203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452613a-7f1b-4250-b718-b783fde1e1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:35.573996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9452613a-7f1b-4250-b718-b783fde1e1a2 None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:35.575520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9452613a-7f1b-4250-b718-b783fde1e1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:35.596250 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:35.655048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1109/2193] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:14:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:35.688644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:35.690189 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:35.741713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:35.792770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36f160-525b-4693-a8e3-356f8888f532 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:35.800108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:35.800741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:35.801573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:35.801895 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:35.825519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:35.827321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:35.857112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1085/2194] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:14:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:14:36.840766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5ec9-eb82-4ce4-a00c-53348399f13c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:36.841266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5ec9-eb82-4ce4-a00c-53348399f13c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:36.841361 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b5ec9-eb82-4ce4-a00c-53348399f13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:36.861021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6b9b5ec9-eb82-4ce4-a00c-53348399f13c None None] MFA Rules not processed for user `a22a06667e10411e8d8ce92db4536a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:36.862311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6b9b5ec9-eb82-4ce4-a00c-53348399f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:36.931986 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1110/2195] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:42.774862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:42.779076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:42.867122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab608f-eb12-47c1-bd12-e5e608813c29 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:42.872461 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:42.873419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:42.873714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:42.874094 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] PATH_INFO: `/v3/users/d1b7434c8a6b498cb37ca16ee23080ff` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:42.875322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b7434c8a6b498cb37ca16ee23080ff)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:42.877774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:42.980828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b7434c8a6b498cb37ca16ee23080ff'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:42.991436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b7434c8a6b498cb37ca16ee23080ff'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:43.030686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b7434c8a6b498cb37ca16ee23080ff'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:43.031656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b7434c8a6b498cb37ca16ee23080ff'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:43.032387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b7434c8a6b498cb37ca16ee23080ff'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:43.080943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1086/2196] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:42 2022] DELETE /identity/v3/users/d1b7434c8a6b498cb37ca16ee23080ff => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:43.086705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:43.087225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:43.087320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:43.115420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:43.116647 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:43.203029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1111/2197] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:43.255673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:43.257425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:43.371308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:43.393639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdc0f1-6264-400a-8755-3030d38a92bc admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:43.399710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf03603-33d0-4e50-9bf0-07f828f890f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:43.400578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf03603-33d0-4e50-9bf0-07f828f890f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:43.400921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf03603-33d0-4e50-9bf0-07f828f890f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:43.401309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf03603-33d0-4e50-9bf0-07f828f890f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:43.412498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf03603-33d0-4e50-9bf0-07f828f890f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:43.414215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf03603-33d0-4e50-9bf0-07f828f890f8 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:43.425487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1087/2198] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:43.844297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:43.846916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:43.864138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc13a99-1d29-4cab-a5d4-299d1a681b01 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:43.874340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:43.875279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:43.875809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:43.876059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] PATH_INFO: `/v3/projects/fcef5c60d21344779c78c9c453e2c798` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:43.884692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcef5c60d21344779c78c9c453e2c798)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:43.892046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:43.959775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcef5c60d21344779c78c9c453e2c798', 'request_id': 'req-ba26bb7f-d6da-487f-bdb2-8464598e16df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:43.969356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcef5c60d21344779c78c9c453e2c798', 'request_id': 'req-ba26bb7f-d6da-487f-bdb2-8464598e16df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:43.977096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcef5c60d21344779c78c9c453e2c798', 'request_id': 'req-ba26bb7f-d6da-487f-bdb2-8464598e16df'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:43.982087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:44.002106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] The token cache is being invalidate because project fcef5c60d21344779c78c9c453e2c798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:44.002756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:44.003575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ba26bb7f-d6da-487f-bdb2-8464598e16df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:44.006166 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1112/2199] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:43 2022] DELETE /identity/v3/projects/fcef5c60d21344779c78c9c453e2c798 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:44.011259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddf686-a719-4781-b884-3f1f4587289b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.011813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddf686-a719-4781-b884-3f1f4587289b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.011813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddf686-a719-4781-b884-3f1f4587289b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.042161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b8ddf686-a719-4781-b884-3f1f4587289b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:44.043389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b8ddf686-a719-4781-b884-3f1f4587289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:44.132974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1088/2200] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:44.141292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb7810-3f2e-4032-a636-5e3602cd1f30 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.141916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb7810-3f2e-4032-a636-5e3602cd1f30 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.142144 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb7810-3f2e-4032-a636-5e3602cd1f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.190186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a0eb7810-3f2e-4032-a636-5e3602cd1f30 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:44.190186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a0eb7810-3f2e-4032-a636-5e3602cd1f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:44.200832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1113/2201] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:14:44.208516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddf686-a719-4781-b884-3f1f4587289b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:44.220214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddf686-a719-4781-b884-3f1f4587289b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:44.226152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:44.226776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.227110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.227453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.230050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:14:44.230701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:44.233139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:44.255060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1089/2202] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:14:44.259117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb7810-3f2e-4032-a636-5e3602cd1f30 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:44.274868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb7810-3f2e-4032-a636-5e3602cd1f30 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:44.282349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:44.284742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.285232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.285761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.287068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:44.290043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:44.346371 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1114/2203] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:14:44 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:14:44.350609 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:44.363170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee717a55-1d5e-4077-9f60-71713e193eca admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:44.488220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:44.489184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.493315 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.493652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.494489 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:44.496683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:44.545256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:44.545483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1090/2204] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:14:44 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:14:44.557542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d4df-d6e9-45c6-bf42-20f2a6ec8772 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:44.562913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:44.564170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.565556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.565899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.566652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:44.568715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:44.583332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1115/2205] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:14:44.585218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:44.597388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d547359-e65a-407f-a8c2-aa36738be673 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:44.602746 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c7ae9-e6fe-448e-a8bf-4c8d38c55307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:44.603390 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c7ae9-e6fe-448e-a8bf-4c8d38c55307 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.603749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c7ae9-e6fe-448e-a8bf-4c8d38c55307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.603962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c7ae9-e6fe-448e-a8bf-4c8d38c55307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.604632 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c7ae9-e6fe-448e-a8bf-4c8d38c55307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:44.606926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c7ae9-e6fe-448e-a8bf-4c8d38c55307 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:44.622959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1091/2206] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:44.624325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:44.636508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907cd3f-36b3-44ad-9468-ecc27a5e5473 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:44.648563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:44.649305 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.649587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.649957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] PATH_INFO: `/v3/projects/f2012eb4cc4c48e5ab703ff0e05eacd4/users/d319ec54156e4168b5ce223a5cd780af/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.672568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2012eb4cc4c48e5ab703ff0e05eacd4, user_id=d319ec54156e4168b5ce223a5cd780af, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:44.676291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:44.699880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1116/2207] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:44 2022] PUT /identity/v3/projects/f2012eb4cc4c48e5ab703ff0e05eacd4/users/d319ec54156e4168b5ce223a5cd780af/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:44.711793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70823d1f-829d-40b9-a032-0e19c5e4b466 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.712066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70823d1f-829d-40b9-a032-0e19c5e4b466 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.712364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70823d1f-829d-40b9-a032-0e19c5e4b466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.749327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-70823d1f-829d-40b9-a032-0e19c5e4b466 None None] MFA Rules not processed for user `d319ec54156e4168b5ce223a5cd780af`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:44.750733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-70823d1f-829d-40b9-a032-0e19c5e4b466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:44.794088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:44.844752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1092/2208] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:44.853063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:44.855942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:44.953369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77015d59-a6bd-4fe7-b340-36500ceee856 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:44.973285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182854e-578f-49a7-8b19-7eae63497351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:44.974140 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182854e-578f-49a7-8b19-7eae63497351 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:44.974557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182854e-578f-49a7-8b19-7eae63497351 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:44.974908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182854e-578f-49a7-8b19-7eae63497351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:44.989424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182854e-578f-49a7-8b19-7eae63497351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:44.990931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182854e-578f-49a7-8b19-7eae63497351 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:45.000933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1117/2209] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:48.265277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70823d1f-829d-40b9-a032-0e19c5e4b466 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:48.334962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70823d1f-829d-40b9-a032-0e19c5e4b466 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:48.339492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:48.340115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:48.340393 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:48.340665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] PATH_INFO: `/v3/users/d319ec54156e4168b5ce223a5cd780af` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:48.341798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d319ec54156e4168b5ce223a5cd780af)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:48.343732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:48.390436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd319ec54156e4168b5ce223a5cd780af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:48.398111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd319ec54156e4168b5ce223a5cd780af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:48.410404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd319ec54156e4168b5ce223a5cd780af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:48.411136 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd319ec54156e4168b5ce223a5cd780af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:48.411766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd319ec54156e4168b5ce223a5cd780af'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:48.426048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1093/2210] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:14:48 2022] DELETE /identity/v3/users/d319ec54156e4168b5ce223a5cd780af => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:48.430643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:48.431143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:48.431261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:48.451365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:48.452612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:48.514980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1118/2211] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:14:48.531878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:48.541996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:48.651816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064b161-620d-4149-b6ff-1fba5dd94d8c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:48.656970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e585eb-25e4-49a5-85a1-c4c44a8f79b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:48.659860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e585eb-25e4-49a5-85a1-c4c44a8f79b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:48.660465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e585eb-25e4-49a5-85a1-c4c44a8f79b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:48.661344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e585eb-25e4-49a5-85a1-c4c44a8f79b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:48.679283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e585eb-25e4-49a5-85a1-c4c44a8f79b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:48.681029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e585eb-25e4-49a5-85a1-c4c44a8f79b7 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:48.689779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:48.708841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1094/2212] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:49.617707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:49.619287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:49.631879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549e9fd-7203-4457-8796-de4dfd4befbb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:49.636703 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:49.637333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:49.637607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:49.637904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] PATH_INFO: `/v3/projects/f2012eb4cc4c48e5ab703ff0e05eacd4` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:49.639013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2012eb4cc4c48e5ab703ff0e05eacd4)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:49.640896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:49.687207 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:49.711717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2012eb4cc4c48e5ab703ff0e05eacd4', 'request_id': 'req-ec513f2e-b77d-4367-8d05-b4906edea9c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:49.720866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2012eb4cc4c48e5ab703ff0e05eacd4', 'request_id': 'req-ec513f2e-b77d-4367-8d05-b4906edea9c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:49.731097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2012eb4cc4c48e5ab703ff0e05eacd4', 'request_id': 'req-ec513f2e-b77d-4367-8d05-b4906edea9c5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:49.751103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] The token cache is being invalidate because project f2012eb4cc4c48e5ab703ff0e05eacd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:14:49.752138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:49.753253 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec513f2e-b77d-4367-8d05-b4906edea9c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:14:49.755748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1119/2213] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:14:49 2022] DELETE /identity/v3/projects/f2012eb4cc4c48e5ab703ff0e05eacd4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:14:49.772494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14449ffd-3a5c-448d-92a7-39639a789e40 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:49.772993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14449ffd-3a5c-448d-92a7-39639a789e40 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:49.773404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14449ffd-3a5c-448d-92a7-39639a789e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:49.799096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-14449ffd-3a5c-448d-92a7-39639a789e40 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:49.800323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-14449ffd-3a5c-448d-92a7-39639a789e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:49.858455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1095/2214] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:49.867072 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436e8c-947b-4141-b5c6-4098d65f7972 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:49.867366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436e8c-947b-4141-b5c6-4098d65f7972 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:49.867643 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436e8c-947b-4141-b5c6-4098d65f7972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:49.888602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-84436e8c-947b-4141-b5c6-4098d65f7972 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:49.889827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-84436e8c-947b-4141-b5c6-4098d65f7972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:49.897991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1120/2215] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:49.902647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14449ffd-3a5c-448d-92a7-39639a789e40 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:49.912891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14449ffd-3a5c-448d-92a7-39639a789e40 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:49.917454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:49.918082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:49.918392 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:49.918712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:49.919356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:14:49.919656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:49.921232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:49.940260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1096/2216] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:14:49.944577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436e8c-947b-4141-b5c6-4098d65f7972 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:49.956631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436e8c-947b-4141-b5c6-4098d65f7972 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:49.962557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:49.963092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:49.963373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:49.963670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:49.964327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:49.966291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:49.994431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1121/2217] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:14:49 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:14:49.998267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:50.008797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662020b-510a-4f77-b5a3-98f3bc066046 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:50.069083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:50.069917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:50.070210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:50.070528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:50.071282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:50.073359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:50.096640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1097/2218] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:14:49 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:14:50.101363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:50.114887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79822409-bf0d-4ef3-9f20-64529f5de8d5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:50.119533 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:50.120120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:50.120402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:50.120677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:50.121347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:50.122985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:50.132930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1122/2219] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:50.136693 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:50.146958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a34a6-c46f-4ea2-95f9-44f17179313d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:50.151598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde47d2-4a86-4e2f-b361-fd775620922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:50.152184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde47d2-4a86-4e2f-b361-fd775620922d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:50.152464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde47d2-4a86-4e2f-b361-fd775620922d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:50.152735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde47d2-4a86-4e2f-b361-fd775620922d admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:50.153371 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde47d2-4a86-4e2f-b361-fd775620922d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:50.155008 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde47d2-4a86-4e2f-b361-fd775620922d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:50.166218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1098/2220] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:14:50.170857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:50.181507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1a2fb-80d5-472b-908f-01d44938aff3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:50.186160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:50.186797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:50.187076 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:50.187350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] PATH_INFO: `/v3/projects/f090725d08904bb3ae6a10598777eaa5/users/7479d03b6f9948e3b8a93337c19af60a/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:50.202078 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f090725d08904bb3ae6a10598777eaa5, user_id=7479d03b6f9948e3b8a93337c19af60a, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:50.204755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:50.218284 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1123/2221] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:14:50 2022] PUT /identity/v3/projects/f090725d08904bb3ae6a10598777eaa5/users/7479d03b6f9948e3b8a93337c19af60a/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:14:50.223963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358dc36-befe-44dd-8582-f842c252eb73 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:50.224330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358dc36-befe-44dd-8582-f842c252eb73 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:50.224601 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358dc36-befe-44dd-8582-f842c252eb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:50.247262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3358dc36-befe-44dd-8582-f842c252eb73 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:50.248479 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3358dc36-befe-44dd-8582-f842c252eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:50.318544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1099/2222] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:14:50.329761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:50.331660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:50.391299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b75d93-f1cc-49e6-8161-d086b4fb194a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:50.395922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:50.396516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:50.396786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:50.397050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:50.406403 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:50.408382 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:50.418055 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1124/2223] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:51.298354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b210a43-c4a9-48a8-b611-a80901711dd1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:51.299760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b210a43-c4a9-48a8-b611-a80901711dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:51.300285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b210a43-c4a9-48a8-b611-a80901711dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:51.310245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:51.346865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-4b210a43-c4a9-48a8-b611-a80901711dd1 None None] MFA Rules not processed for user `7479d03b6f9948e3b8a93337c19af60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:51.348100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4b210a43-c4a9-48a8-b611-a80901711dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:51.418036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1100/2224] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:51.428962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:51.431008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:51.492956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c9cf4-85a8-4c66-a3a3-64258fc986a7 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:51.496588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45790a3-e3f0-44af-a5b6-8480e4f2c503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:51.497186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45790a3-e3f0-44af-a5b6-8480e4f2c503 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:51.497646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45790a3-e3f0-44af-a5b6-8480e4f2c503 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:51.497950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45790a3-e3f0-44af-a5b6-8480e4f2c503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:51.507360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45790a3-e3f0-44af-a5b6-8480e4f2c503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:51.509046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45790a3-e3f0-44af-a5b6-8480e4f2c503 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:51.513771 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:14:51.519471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1125/2225] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:14:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:14:52.938104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:52.938744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:52.938744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:52.960240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] MFA Rules not processed for user `7479d03b6f9948e3b8a93337c19af60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:52.961662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:52.968727 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1101/2226] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:53.975988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c2214-12f8-4ce1-8232-6182aeed3528 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:53.976769 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c2214-12f8-4ce1-8232-6182aeed3528 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:53.977156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c2214-12f8-4ce1-8232-6182aeed3528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:54.005595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6e0c2214-12f8-4ce1-8232-6182aeed3528 None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:54.007694 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6e0c2214-12f8-4ce1-8232-6182aeed3528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:54.115607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1126/2227] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:14:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:14:54.140158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:14:54.142413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:54.280434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c83313-027b-4480-b599-c6574a61e1e4 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:14:54.336048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15afafe-491b-4c31-833f-5d7eac19764c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:14:54.340769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15afafe-491b-4c31-833f-5d7eac19764c service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:54.342865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15afafe-491b-4c31-833f-5d7eac19764c service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:54.344787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15afafe-491b-4c31-833f-5d7eac19764c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:54.358888 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15afafe-491b-4c31-833f-5d7eac19764c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:14:54.362310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15afafe-491b-4c31-833f-5d7eac19764c service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:14:54.396948 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1102/2228] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:14:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 14:14:56.878416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b7959-ca25-46c5-9023-7d025cbee3ec None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:14:56.879046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b7959-ca25-46c5-9023-7d025cbee3ec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:14:56.879046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b7959-ca25-46c5-9023-7d025cbee3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:14:56.914692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8a2b7959-ca25-46c5-9023-7d025cbee3ec None None] MFA Rules not processed for user `961012089bd147ecab4fcdf5ea3ae4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:14:56.917018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8a2b7959-ca25-46c5-9023-7d025cbee3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:14:56.999939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1127/2229] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:14:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:15:04.369247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=65 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:15:06.990246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=50 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:15:18.802780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d274ea-5b75-44d9-85ca-c4dab4b35933 None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:18.803380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d274ea-5b75-44d9-85ca-c4dab4b35933 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:18.803713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d274ea-5b75-44d9-85ca-c4dab4b35933 None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:18.805958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1103/2230] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:15:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:15:18.812071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971019-8788-432f-928e-9d8acee403f8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:18.813657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971019-8788-432f-928e-9d8acee403f8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:18.814158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971019-8788-432f-928e-9d8acee403f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:18.838109 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ab971019-8788-432f-928e-9d8acee403f8 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:15:18.839520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab971019-8788-432f-928e-9d8acee403f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:18.928706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1128/2231] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:15:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:15:18.940893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d274ea-5b75-44d9-85ca-c4dab4b35933 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:15:18.950761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-16d274ea-5b75-44d9-85ca-c4dab4b35933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:19.024988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d274ea-5b75-44d9-85ca-c4dab4b35933 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:15:19.028566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc5a27-341d-45d5-94d6-c987e67d63a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7214e6addc646a899efb82283ededc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:15:19.029170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5a27-341d-45d5-94d6-c987e67d63a5 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:19.029452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5a27-341d-45d5-94d6-c987e67d63a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:19.029743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc5a27-341d-45d5-94d6-c987e67d63a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:19.040913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc5a27-341d-45d5-94d6-c987e67d63a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:15:19.042580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc5a27-341d-45d5-94d6-c987e67d63a5 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:15:19.055341 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1104/2232] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:15:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:15:19.124397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854bfc-316a-454d-b15b-2548bd4729e1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:19.124724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854bfc-316a-454d-b15b-2548bd4729e1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:19.125006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45854bfc-316a-454d-b15b-2548bd4729e1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:19.127488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1129/2233] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:15:19.133920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59c94-0e36-4599-9cce-9a3257e0595b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:19.133920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59c94-0e36-4599-9cce-9a3257e0595b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:19.133920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59c94-0e36-4599-9cce-9a3257e0595b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:19.158379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-76d59c94-0e36-4599-9cce-9a3257e0595b None None] MFA Rules not processed for user `11a418a2a35146b59c433421fe507500`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:15:19.159564 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-76d59c94-0e36-4599-9cce-9a3257e0595b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:19.167256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1105/2234] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:15:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:15:19.181331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45854bfc-316a-454d-b15b-2548bd4729e1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:15:19.193547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45854bfc-316a-454d-b15b-2548bd4729e1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:15:19.197200 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92603cf8-497c-422f-ab44-311a9a2ae74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:15:19.197942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603cf8-497c-422f-ab44-311a9a2ae74e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:19.198223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603cf8-497c-422f-ab44-311a9a2ae74e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:19.198494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92603cf8-497c-422f-ab44-311a9a2ae74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:19.208307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603cf8-497c-422f-ab44-311a9a2ae74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:15:19.209896 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92603cf8-497c-422f-ab44-311a9a2ae74e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:15:19.218156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1130/2235] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:15:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:15:19.307468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c012c-5b57-49e3-9cb1-cd736300414c None None] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:19.307820 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c012c-5b57-49e3-9cb1-cd736300414c None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:19.308151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c012c-5b57-49e3-9cb1-cd736300414c None None] PATH_INFO: `/` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:19.309698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1106/2236] 10.222.0.20 () {54 vars in 921 bytes} [Tue Sep 13 14:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 14:15:19.315638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:19.315917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:19.316185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:19.336232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] MFA Rules not processed for user `6a4ea803f6e44648bb217abd00e11c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:15:19.337428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:19.348584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1131/2237] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Sep 13 14:15:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:15:19.359314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c012c-5b57-49e3-9cb1-cd736300414c None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:15:19.370807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c012c-5b57-49e3-9cb1-cd736300414c None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:15:19.374343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:15:19.374923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:19.375241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:19.375572 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:19.385950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:15:19.387422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:15:19.398400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1107/2238] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:15:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:15:22.330531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:15:22.333919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:22.428199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f6e06-b641-4371-90d6-6a883ec3e649 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:15:22.433204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138222df-8c5f-4e48-84a9-46444c059536 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:15:22.433875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138222df-8c5f-4e48-84a9-46444c059536 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:22.434124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138222df-8c5f-4e48-84a9-46444c059536 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:22.434393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138222df-8c5f-4e48-84a9-46444c059536 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:22.436230 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-138222df-8c5f-4e48-84a9-46444c059536 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:22.448370 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138222df-8c5f-4e48-84a9-46444c059536 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:15:22.450028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138222df-8c5f-4e48-84a9-46444c059536 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:15:22.466092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1132/2239] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:15:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:15:24.823827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:15:24.844501 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ba378-4b0a-415b-9e92-7ca913c85706 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:15:24.853091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814193f-fbd3-476a-a431-3679553d04ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:15:24.853718 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814193f-fbd3-476a-a431-3679553d04ab service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:24.854568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814193f-fbd3-476a-a431-3679553d04ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:24.854854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814193f-fbd3-476a-a431-3679553d04ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:24.856735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a814193f-fbd3-476a-a431-3679553d04ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:24.967331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814193f-fbd3-476a-a431-3679553d04ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:15:24.969049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814193f-fbd3-476a-a431-3679553d04ab service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:15:24.982317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1108/2240] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:15:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:15:30.446065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c32b0-bd1f-4479-8fc9-8a45bb0803df None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:30.446517 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c32b0-bd1f-4479-8fc9-8a45bb0803df None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:30.446714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c32b0-bd1f-4479-8fc9-8a45bb0803df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:30.467641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-064c32b0-bd1f-4479-8fc9-8a45bb0803df None None] MFA Rules not processed for user `7479d03b6f9948e3b8a93337c19af60a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:15:30.469886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-064c32b0-bd1f-4479-8fc9-8a45bb0803df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:30.477105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1133/2241] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:15:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:15:32.430251 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75db9d-355e-46ca-907d-183015ff5a2a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:32.431018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75db9d-355e-46ca-907d-183015ff5a2a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:32.431456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75db9d-355e-46ca-907d-183015ff5a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:32.463607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9a75db9d-355e-46ca-907d-183015ff5a2a None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:15:32.464819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9a75db9d-355e-46ca-907d-183015ff5a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:32.472509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1109/2242] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:15:32.486199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c32b0-bd1f-4479-8fc9-8a45bb0803df None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:15:32.499588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c32b0-bd1f-4479-8fc9-8a45bb0803df None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:15:32.513630 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54821a-0edd-4bfa-a2f1-02a583ea2243 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:15:32.515983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54821a-0edd-4bfa-a2f1-02a583ea2243 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:32.516566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54821a-0edd-4bfa-a2f1-02a583ea2243 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:32.516987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54821a-0edd-4bfa-a2f1-02a583ea2243 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:32.531705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54821a-0edd-4bfa-a2f1-02a583ea2243 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:15:32.534956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54821a-0edd-4bfa-a2f1-02a583ea2243 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:15:32.549228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1134/2243] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:15:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:15:42.446879 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=68 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:15:42.541556 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:15:43.274994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe518-fe7e-4d05-a785-b6b89aa2c11a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:43.275277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe518-fe7e-4d05-a785-b6b89aa2c11a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:43.275597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fe518-fe7e-4d05-a785-b6b89aa2c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:43.297717 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-eb8fe518-fe7e-4d05-a785-b6b89aa2c11a None None] MFA Rules not processed for user `3e4eccc7ece24cf4bcfee361d3342a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:15:43.298962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-eb8fe518-fe7e-4d05-a785-b6b89aa2c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:43.358234 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1110/2244] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:15:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:15:53.352099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=25 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:15:58.721289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cfde5-28ea-48d4-8c42-160773bafd18 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:58.721883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cfde5-28ea-48d4-8c42-160773bafd18 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:58.721883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cfde5-28ea-48d4-8c42-160773bafd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:58.744101 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c13cfde5-28ea-48d4-8c42-160773bafd18 None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:15:58.745320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c13cfde5-28ea-48d4-8c42-160773bafd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:15:58.754562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1135/2245] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 13 14:15:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:15:58.786767 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe518-fe7e-4d05-a785-b6b89aa2c11a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:15:58.801483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fe518-fe7e-4d05-a785-b6b89aa2c11a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:15:58.806855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:15:58.808044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:15:58.808647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:15:58.809005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:15:58.819974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:15:58.821640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:15:58.830344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1111/2246] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:15:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:16:08.738107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:16:08.826786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:16:14.069505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:16:14.070102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:16:14.070102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:16:14.090823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:16:14.092538 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:16:14.099829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1136/2247] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Sep 13 14:16:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:16:14.112502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:16:14.123169 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de4301-9d4b-461b-90e7-6dcc9f413563 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:16:14.128568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:16:14.129172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:16:14.129455 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:16:14.129748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:16:14.140098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:16:14.140763 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:16:14.153152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1112/2248] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:16:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:16:24.084717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:16:24.149016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:17:17.308460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:17.310136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:17.325416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a60ffd-33b2-4209-beaf-2644cf6d434f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:17.331219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:17.331821 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:17.332099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:17.332394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:17.334132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:17.395051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:17.396669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:17.406484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1137/2249] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:24.575081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:24.585057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c5ba3-60a2-4fe2-aa29-9627e8fa7ce6 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:24.588697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:24.589279 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:24.589554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:24.589864 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:24.591803 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:24.650144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:24.651556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:24.660731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1113/2250] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:26.412050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:26.422112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022dbae-c908-452a-8d33-2ad5a9bd10a3 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:26.425792 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:26.426278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:26.426554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:26.426822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:26.428586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:26.439520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:26.440977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:26.450512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1138/2251] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:33.926342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:33.928067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:33.944170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ca7bb-ba29-4dbf-8b99-1cc4fe1bf54b service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:33.949815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:33.950417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:33.950803 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:33.951167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:33.953001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:34.012545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:34.014175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:34.023370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1114/2252] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:36.445812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=34 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:17:44.019282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=56 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:17:47.326117 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:47.327605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:47.339369 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c30bca-810f-43a6-850c-21538e24b348 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:47.343616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:47.344223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:47.344501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:47.344772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:47.346550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:47.360831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:47.362215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:47.373692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1139/2253] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:47.891759 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:47.902291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba21c8-27c8-455f-b36d-07890204377b service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:47.906426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:47.907031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:47.907311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:47.907584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:47.909416 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:47.919827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:47.921411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:47.930199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1115/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:49.474762 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:49.484411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c73f8f-9014-4147-ab53-5112130bda99 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:49.488062 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:49.488677 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:49.488957 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:49.489446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:49.491249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:49.507201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:49.508665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:49.521712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1140/2255] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:49.881836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:17:49.896848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0070049-1e50-4f14-97a2-35b885b80521 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:17:49.906934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:17:49.906934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:17:49.906934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:17:49.906934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:17:49.907514 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:17:49.919906 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:17:49.921628 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:17:49.933070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1116/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:17:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:17:59.515923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=12 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:17:59.927865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=12 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:18:38.864693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:18:38.873796 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99535-0609-4b4a-919b-a2178c94f756 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:18:38.877510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:18:38.878146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:18:38.878424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:18:38.878699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:18:38.880469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:18:38.891481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:18:38.892893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:18:38.902013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1141/2257] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:18:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:18:48.897990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:19:19.178296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:19:19.190303 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8443559-b655-4248-9404-9e4d871c35d0 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:19:19.195792 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:19:19.196384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:19:19.196661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:19:19.196933 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:19:19.198656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:19:19.256520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:19:19.258131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:19:19.267824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1117/2258] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:19:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:19:29.262852 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=28 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:19:51.555171 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:19:51.556585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:19:51.567839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1305131-19d4-4c64-9400-2627080a7ec0 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:19:51.571463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:19:51.572048 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:19:51.572325 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:19:51.572600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:19:51.581335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:19:51.582823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:19:51.591613 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1142/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:19:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:20:01.587624 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:20:22.913610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:20:22.915034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:20:23.000127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d21cb-1569-4ff5-adcd-eda13f55fcbf service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:20:23.005550 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:20:23.006190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:20:23.006473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:20:23.006744 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:20:23.015162 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:20:23.016712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:20:23.026941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1118/2260] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:20:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:20:33.022657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=37 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:22:16.978622 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:17.033024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff148ee-82fb-44d8-88fd-1430539eeabf service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:17.038946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:17.039537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:17.039818 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:17.040082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:17.047841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:17.049394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:17.076488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1143/2261] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:22:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:22:24.454133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:24.467294 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887d32f-1431-40cb-b596-258654d2a4e7 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:24.473305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c11341-3263-4354-b3af-f3f9beaa034d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:24.473484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c11341-3263-4354-b3af-f3f9beaa034d service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:24.473713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c11341-3263-4354-b3af-f3f9beaa034d service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:24.474074 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c11341-3263-4354-b3af-f3f9beaa034d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:24.482122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c11341-3263-4354-b3af-f3f9beaa034d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:24.483678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c11341-3263-4354-b3af-f3f9beaa034d service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:24.511999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1119/2262] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:22:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:22:26.179003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:26.205938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e92d9-4222-416b-8458-ad0ea1fe688a service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:26.210460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:26.211053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:26.211329 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:26.211599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:26.220106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:26.221519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:26.248363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1144/2263] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:22:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:22:30.602968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94801388-ce05-4a93-adb6-7ddc4882a21b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:30.603430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94801388-ce05-4a93-adb6-7ddc4882a21b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:30.603540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94801388-ce05-4a93-adb6-7ddc4882a21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:30.664867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-94801388-ce05-4a93-adb6-7ddc4882a21b None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:22:30.666114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-94801388-ce05-4a93-adb6-7ddc4882a21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:22:30.682486 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1120/2264] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 13 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:22:30.728029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:30.748834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b08ef-fa40-4d74-a388-1ed85d404031 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:30.753846 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:30.754444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:30.754722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:30.755002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:30.762902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:30.764469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:30.773308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1145/2265] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:22:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:22:31.159808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94801388-ce05-4a93-adb6-7ddc4882a21b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:31.179579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94801388-ce05-4a93-adb6-7ddc4882a21b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:31.183417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:31.184017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:31.184298 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:31.184568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:31.193937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:31.195355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:31.222704 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1121/2266] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:22:31.663276 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:31.674855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d61258-e753-483f-89c5-bca56902e11f service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:31.680386 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:31.680982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:31.681295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:31.681581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:31.683346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:22:31.695755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:31.697439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:31.709655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1146/2267] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:22:31.716809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:31.727059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b55608-0676-4341-a4fe-5669b0815f5c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:31.732099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:31.732738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:31.733109 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:31.733650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] PATH_INFO: `/v3/users/7479d03b6f9948e3b8a93337c19af60a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:31.735332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7479d03b6f9948e3b8a93337c19af60a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:31.737297 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:31.793018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7479d03b6f9948e3b8a93337c19af60a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:31.799837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7479d03b6f9948e3b8a93337c19af60a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:31.811927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7479d03b6f9948e3b8a93337c19af60a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:31.812595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7479d03b6f9948e3b8a93337c19af60a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:31.813210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7479d03b6f9948e3b8a93337c19af60a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:31.826826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1122/2268] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:22:31 2022] DELETE /identity/v3/users/7479d03b6f9948e3b8a93337c19af60a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:22:32.002737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:32.014902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e792165-6f7b-497c-8261-d904fe0a4dbf service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:32.075462 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:32.076152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:32.076431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:32.076742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] PATH_INFO: `/v3/projects/f090725d08904bb3ae6a10598777eaa5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:32.077920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f090725d08904bb3ae6a10598777eaa5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:32.079882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:32.102524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:22:32.125197 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f090725d08904bb3ae6a10598777eaa5', 'request_id': 'req-11b808d1-2d1f-4382-94ed-1c678dd92fc4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:32.131312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f090725d08904bb3ae6a10598777eaa5', 'request_id': 'req-11b808d1-2d1f-4382-94ed-1c678dd92fc4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:32.137322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f090725d08904bb3ae6a10598777eaa5', 'request_id': 'req-11b808d1-2d1f-4382-94ed-1c678dd92fc4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:32.162505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] The token cache is being invalidate because project f090725d08904bb3ae6a10598777eaa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:22:32.162793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:32.163484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:22:32.164968 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1147/2269] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:22:32 2022] DELETE /identity/v3/projects/f090725d08904bb3ae6a10598777eaa5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:22:34.603076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:22:34.604532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:22:34.626295 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:22:34.690043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a2320-ff2d-499e-8d2f-49d6d42e72e1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:22:34.754685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:22:34.755425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:22:34.755716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:22:34.755989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:22:34.757781 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:22:34.837044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:22:34.838622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:22:34.856290 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1123/2270] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:22:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:22:42.159619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:22:44.853678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:23:07.735134 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:23:07.736705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:23:07.816194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b808d1-2d1f-4382-94ed-1c678dd92fc4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:23:07.819996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:23:07.820612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:23:07.820895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:23:07.821167 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:23:07.823010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:23:07.837590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:23:07.839045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:23:07.849713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1148/2271] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:23:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:23:11.628048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:23:11.629473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:23:11.641152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a7f22-9ef3-445e-9a7e-818931523548 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:23:11.644932 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:23:11.645519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:23:11.645827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:23:11.646100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:23:11.647720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:23:11.660470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:23:11.661896 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:23:11.672533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1124/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:23:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:23:17.844951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=31 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:23:21.667770 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:23:24.603552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:23:24.613599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d16da0-ebde-4de1-bd42-e11b580d19dd service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:23:24.617441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:23:24.618147 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:23:24.618492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:23:24.618835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:23:24.621240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:23:24.631201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:23:24.632726 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:23:24.646553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1149/2273] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:23:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:23:34.638554 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:23:39.726253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:23:39.727634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:23:39.793485 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be587-f2de-45e6-ba12-8273fa231598 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:23:39.797629 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:23:39.798247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:23:39.798522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:23:39.798790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:23:39.800512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:23:39.878337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:23:39.879737 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:23:39.898176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1125/2274] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:23:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:23:49.895491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=66 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:24:19.126312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:24:19.135549 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ea8ae-d458-439a-adb2-5607ced35c8b service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:24:19.139126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:24:19.139717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:24:19.140001 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:24:19.140278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:24:19.142315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:24:19.230656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:24:19.232082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:24:19.249766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1150/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:24:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:24:29.247223 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=41 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:25:28.854548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:25:28.856007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:25:28.926344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d435648-b2a4-46f2-b01d-e8e035b6d120 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:25:28.931712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:25:28.932317 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:25:28.932597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:25:28.932869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:25:28.934606 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:25:28.945961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:25:28.947521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:25:28.957356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1126/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:25:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:25:38.953127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=32 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:27:17.628522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:27:17.629951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:27:17.640700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bae7c-c777-46b1-a269-6dcd9bfd4b40 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:27:17.644324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:27:17.645158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:27:17.645434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:27:17.645704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:27:17.647352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:27:17.716312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:27:17.717755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:27:17.727358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1151/2277] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:27:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:27:24.167080 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:27:24.168661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:27:24.183802 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622bbea-e956-4535-ad4a-010729d4b5c7 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:27:24.189423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:27:24.190083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:27:24.190362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:27:24.190636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:27:24.192337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:27:24.264956 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:27:24.266749 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:27:24.276917 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1127/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:27:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:27:26.723551 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:27:26.725256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:27:26.741121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0c132-575b-4372-a997-43ee75dbaae7 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:27:26.746854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:27:26.747446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:27:26.747721 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:27:26.747990 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:27:26.749692 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:27:26.819545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:27:26.821127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:27:26.830759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1152/2279] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:27:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:27:34.272573 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=33 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:27:35.399831 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:27:35.409947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f5a2c-3897-484b-9814-d4ea66d6160d service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:27:35.413618 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:27:35.414249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:27:35.414637 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:27:35.414914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:27:35.423205 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:27:35.424854 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:27:35.433645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1128/2280] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:27:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:27:36.825826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=66 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:27:45.429563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:28:11.461145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:28:11.470373 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dc00c-c5d1-4580-9c81-89f92d9119b7 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:28:11.475122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:28:11.475715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:28:11.475992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:28:11.476265 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:28:11.486306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:28:11.487906 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:28:11.498674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1153/2281] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:28:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:28:19.724814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:28:19.734041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a05675-6f1c-4395-9774-275de202c0a0 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:28:19.737858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:28:19.738505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:28:19.738788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:28:19.739062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:28:19.749084 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:28:19.750515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:28:19.761777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1129/2282] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:28:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:28:21.494029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:28:28.537010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:28:28.545720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815d676-0053-44ce-9c38-98879a0db9b6 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:28:28.549339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:28:28.549964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:28:28.550262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:28:28.550531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:28:28.557852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:28:28.559266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:28:28.567696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1154/2283] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:28:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:28:29.757040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:28:38.563657 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:28:40.666663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:28:40.677975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d08a-8227-4653-b022-4d2449304abb service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:28:40.684301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:28:40.684966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:28:40.685243 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:28:40.685507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:28:40.694351 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:28:40.695974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:28:40.704822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1130/2284] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:28:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:28:50.700731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:29:09.312531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:29:09.327566 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08ae20-f0ab-47b6-a672-3278e76f4b2e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:29:09.333848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:29:09.334576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:29:09.334854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:29:09.335123 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:29:09.336862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:29:09.350286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:29:09.351879 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:29:09.372964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1155/2285] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:29:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:29:19.358258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:29:20.030686 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:29:20.039858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1573d-a02f-4ecb-b4b9-d8d7fe3ba823 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:29:20.043406 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:29:20.043979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:29:20.044259 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:29:20.044528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:29:20.053192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:29:20.054649 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:29:20.063304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1131/2286] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:29:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:29:30.059380 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:30:31.898930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:30:31.934947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1080a-4664-48d3-9dd8-dd13835c15e4 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:30:31.940084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:30:31.940679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:30:31.940955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:30:31.941227 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:30:31.949471 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:30:31.951065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:30:31.961617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1156/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:30:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:30:41.957183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=16 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:32:18.088659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:32:18.108562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e40611-aae6-4c10-97f2-bb051d1d3fe0 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:32:18.112293 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:32:18.113019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:32:18.113302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:32:18.114337 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:32:18.138281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:32:18.139644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:32:18.168268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1132/2288] 10.222.0.20 () {60 vars in 1396 bytes} [Tue Sep 13 14:32:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:32:24.253248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:32:24.273612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fe966-209a-40b2-86d2-e36f16d109a6 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:32:24.277277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:32:24.277913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:32:24.278192 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:32:24.278464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:32:24.287044 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:32:24.289196 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:32:24.298456 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1157/2289] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:32:27.545584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:32:27.565148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651d53a-c4bd-40ab-a121-27e318b07ca0 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:32:27.568812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:32:27.569961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:32:27.570236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:32:27.570543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:32:27.579919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:32:27.581639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:32:27.609997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1133/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:32:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:32:34.294082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=11 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:32:35.671606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:32:35.762248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c529-23ac-4214-b1a4-03ac01f5c67d service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:32:35.767655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:32:35.768249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:32:35.768525 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:32:35.768793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:32:35.770683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:32:35.859081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:32:35.860658 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:32:35.879542 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1158/2291] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:32:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:32:37.607187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=41 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:32:45.876954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=80 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:33:11.335451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:33:11.400394 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09138b-8eef-4f04-936e-30fd943f6b8b service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:33:11.404082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:33:11.404683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:33:11.404965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:33:11.405239 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:33:11.416355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:33:11.417824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:33:11.429422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1134/2292] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:33:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:33:19.591776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:33:19.593377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:33:19.616909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5550c53-8bab-4836-9a67-59a6793aea45 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:33:19.620526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:33:19.621107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:33:19.621382 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:33:19.621648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:33:19.623443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:33:19.649850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:33:19.651707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:33:19.662998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1159/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:33:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:33:21.424193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=30 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:33:29.658075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=16 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:33:33.515832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:33:33.525041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf29613-4c3c-4623-9ef0-362dab27f5f3 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:33:33.528733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:33:33.529329 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:33:33.529617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:33:33.529925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:33:33.531829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:33:33.542608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:33:33.544024 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:33:33.552155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1135/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:33:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:33:40.259416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:33:40.268043 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc2c01-f079-4342-93b7-c9b8447a7225 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:33:40.271722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:33:40.272578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:33:40.272858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:33:40.273132 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:33:40.276023 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:33:40.355636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:33:40.357083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:33:40.375451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1160/2295] 10.222.0.20 () {60 vars in 1396 bytes} [Tue Sep 13 14:33:40 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:33:43.548517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:33:50.372803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=41 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:34:18.418348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:34:18.427395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f1280-53bb-4ae6-83c3-dc7cf36650af service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:34:18.431735 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d0271-59d0-4d5f-9a2b-1b8e03387549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:34:18.432339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d0271-59d0-4d5f-9a2b-1b8e03387549 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:34:18.432617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d0271-59d0-4d5f-9a2b-1b8e03387549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:34:18.432889 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d0271-59d0-4d5f-9a2b-1b8e03387549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:34:18.441347 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d0271-59d0-4d5f-9a2b-1b8e03387549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:34:18.443029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d0271-59d0-4d5f-9a2b-1b8e03387549 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:34:18.453045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1136/2296] 10.222.0.20 () {60 vars in 1396 bytes} [Tue Sep 13 14:34:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:34:20.731680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:34:20.747103 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753bbc-58bb-4573-8c73-2063f33ade55 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:34:20.753619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:34:20.754447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:34:20.754907 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:34:20.755180 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:34:20.757028 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:34:20.770609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:34:20.772875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:34:20.801059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1161/2297] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:34:28.448566 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:34:30.782817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:34:55.908325 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf917303-9151-4492-914c-e37ab26e5246 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:34:55.908878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf917303-9151-4492-914c-e37ab26e5246 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:34:55.908878 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf917303-9151-4492-914c-e37ab26e5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:34:55.942173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cf917303-9151-4492-914c-e37ab26e5246 None None] MFA Rules not processed for user `92c30fba107b4dfbb7b7b1582cee2ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:34:55.943801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cf917303-9151-4492-914c-e37ab26e5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:34:55.951148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1137/2298] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:34:56.497352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:34:56.498954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:34:56.512307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42ea6d-9d2f-4264-8a7b-b07f0a7f4bc2 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:34:56.515882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:34:56.516469 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:34:56.516745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:34:56.517011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:34:56.518754 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:34:56.529631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:34:56.531104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:34:56.539082 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1162/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:34:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:34:56.888104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf917303-9151-4492-914c-e37ab26e5246 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:34:56.900611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf917303-9151-4492-914c-e37ab26e5246 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:34:56.905579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:34:56.906219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:34:56.906679 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:34:56.906953 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:34:56.908684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:34:56.934521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:34:56.936089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:34:56.946229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1138/2300] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:34:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:35:05.746970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:35:05.770502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aacc0-5696-44ed-bf9b-87d7c69c1afc service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:35:05.777539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8a7c8-b3a6-4474-8df7-fbd64d7a1d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:35:05.777922 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a7c8-b3a6-4474-8df7-fbd64d7a1d30 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:35:05.778201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a7c8-b3a6-4474-8df7-fbd64d7a1d30 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:35:05.778474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a7c8-b3a6-4474-8df7-fbd64d7a1d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:35:05.780260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-acf8a7c8-b3a6-4474-8df7-fbd64d7a1d30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:35:05.795234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8a7c8-b3a6-4474-8df7-fbd64d7a1d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:35:05.796809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8a7c8-b3a6-4474-8df7-fbd64d7a1d30 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:35:05.807936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1163/2301] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:35:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:35:06.941700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=17 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:35:09.883134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:35:09.895970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24434023-db39-47dc-b80e-ceb03d0f29af service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:35:09.902236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:35:09.902833 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:35:09.903110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:35:09.903382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:35:09.921777 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:35:09.923364 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:35:09.934685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1139/2302] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:35:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:35:13.814882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a446-5710-4d38-9524-eb4c247a91e3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:35:13.815356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a446-5710-4d38-9524-eb4c247a91e3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:35:13.815463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a446-5710-4d38-9524-eb4c247a91e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:35:13.843878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b879a446-5710-4d38-9524-eb4c247a91e3 None None] MFA Rules not processed for user `a22a06667e10411e8d8ce92db4536a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:35:13.845063 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b879a446-5710-4d38-9524-eb4c247a91e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:35:13.851882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1164/2303] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:35:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:35:19.929600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:35:23.841584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=18 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:35:36.449335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:35:36.450774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:35:36.518730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2806ff-545a-45a1-a39a-c3dca998e649 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:35:36.524121 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:35:36.524729 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:35:36.525011 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:35:36.525313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:35:36.527934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:35:36.539977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:35:36.541554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:35:36.551722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1140/2304] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:35:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:35:46.547498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=32 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:35:50.946041 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aa9cb-46db-4ee7-afcf-9b2dcd9751e5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:35:50.946482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aa9cb-46db-4ee7-afcf-9b2dcd9751e5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:35:50.946588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65aa9cb-46db-4ee7-afcf-9b2dcd9751e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:35:51.006164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f65aa9cb-46db-4ee7-afcf-9b2dcd9751e5 None None] MFA Rules not processed for user `929edc18c5874105a349dcc2b4cad14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:35:51.007385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f65aa9cb-46db-4ee7-afcf-9b2dcd9751e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:35:51.070181 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1165/2305] 10.222.0.20 () {58 vars in 967 bytes} [Tue Sep 13 14:35:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:35:51.118332 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:35:51.119869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:35:51.207363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80943e4c-1815-4ca6-b936-33aeea4d0977 service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:35:51.212348 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15e78b0e48a141edad05f3c064ac04f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:35:51.212941 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:35:51.213219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:35:51.214062 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:35:51.222155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:35:51.223773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:35:51.232339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1141/2306] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Sep 13 14:35:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 14:35:51.777186 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65aa9cb-46db-4ee7-afcf-9b2dcd9751e5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:35:51.786497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65aa9cb-46db-4ee7-afcf-9b2dcd9751e5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:35:51.790067 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:35:51.790671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:35:51.790953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:35:51.791226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:35:51.793178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:35:51.877824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:35:51.879295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:35:51.898710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1166/2307] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:36:01.228499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=41 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:36:01.895807 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:37:24.390814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:37:24.392375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:37:24.408714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194b9e4-7606-479d-98af-9206747a2581 service swift] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:37:24.414156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:37:24.414743 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:37:24.415020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:37:24.415289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:37:24.417039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:37:24.496021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:37:24.497712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:37:24.516262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1142/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:37:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:37:34.513231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=41 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:38:13.343344 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:38:13.351930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0395bbc-11fa-44d2-b6dd-dfef056f7394 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:38:13.356178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:38:13.356778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:38:13.357061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:38:13.357337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:38:13.359204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:38:13.372963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:38:13.374454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:38:13.385318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1167/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:38:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:38:21.113118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:38:21.123246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc17d7b-7fd8-498a-81aa-653b800c5ac8 service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:38:21.126903 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:38:21.127495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:38:21.127778 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:38:21.128050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:38:21.139054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:38:21.140468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:38:21.151435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1143/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:38:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:38:21.393235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:38:21.394824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:38:21.462242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca89daf-c551-4a45-bea7-90ea0569263b service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:38:21.465903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:38:21.466498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:38:21.466775 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:38:21.467045 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:38:21.468820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:38:21.542291 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:38:21.544266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:38:21.553446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1168/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:38:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:38:21.596101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:38:21.605154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b244bcc-37a6-44f5-9286-986ad028207c service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:38:21.609500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:38:21.610125 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:38:21.610430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:38:21.610702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:38:21.612503 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:38:21.681927 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:38:21.683276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:38:21.692402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1144/2312] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:38:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:38:21.735532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:38:21.743893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68799a2-1f26-4221-affe-5afbc9bfb2d6 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:38:21.747485 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:38:21.748084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:38:21.748345 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:38:21.748616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:38:21.757941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:38:21.759358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:38:21.768204 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1169/2313] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:38:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:38:31.688264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=39 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:38:31.764065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=70 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:38:36.560200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:38:36.569381 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a0047-0c0a-411e-9075-2965ad02513d service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:38:36.572989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafcc73-825e-4554-bcef-b08ff46ee2b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:38:36.573616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafcc73-825e-4554-bcef-b08ff46ee2b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:38:36.573938 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafcc73-825e-4554-bcef-b08ff46ee2b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:38:36.574216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafcc73-825e-4554-bcef-b08ff46ee2b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:38:36.581826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafcc73-825e-4554-bcef-b08ff46ee2b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:38:36.583249 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafcc73-825e-4554-bcef-b08ff46ee2b4 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:38:36.591920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1145/2314] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:38:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:38:46.587622 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:39:09.218069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:39:09.220515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:39:09.231814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73806144-dc2a-4343-8659-7cc0c3d95335 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:39:09.235817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:39:09.236392 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:39:09.236669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:39:09.236940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:39:09.245446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:39:09.246904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:39:09.255826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1170/2315] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:39:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:39:19.251690 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:40:21.782596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef395a1-9568-4f2b-90e5-d54c6ab60740 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:40:21.783140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef395a1-9568-4f2b-90e5-d54c6ab60740 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:40:21.783140 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef395a1-9568-4f2b-90e5-d54c6ab60740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:40:21.810894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1ef395a1-9568-4f2b-90e5-d54c6ab60740 None None] MFA Rules not processed for user `17e8dbc75366467f8fc6f0681e722751`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:40:21.812100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1ef395a1-9568-4f2b-90e5-d54c6ab60740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:40:21.818930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1146/2316] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:40:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:40:22.130743 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:40:22.140757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353aae2-9758-4ba6-a9c0-ea5c9f1e69cd service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:40:22.144397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:40:22.144989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:40:22.145266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:40:22.145536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:40:22.147523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:40:22.157176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:40:22.158639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:40:22.166416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1171/2317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:40:22.499215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef395a1-9568-4f2b-90e5-d54c6ab60740 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:40:22.511567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef395a1-9568-4f2b-90e5-d54c6ab60740 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:40:22.517012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:40:22.517557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:40:22.517872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:40:22.518153 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:40:22.538265 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:40:22.539943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:40:22.552042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1147/2318] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:40:23.162678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:40:23.164444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:40:23.192767 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3292e6-a360-4c53-8472-9552f91b1a6e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:40:23.198844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:40:23.199522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:40:23.199811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:40:23.200096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:40:23.214351 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:40:23.215934 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:40:23.226849 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1172/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:40:29.050497 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:40:29.063476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9cef4-1a83-4448-b812-4c9492a5192a service placement] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:40:29.068825 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:40:29.069420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:40:29.069696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:40:29.070009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:40:29.079653 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:40:29.081207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:40:29.092594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1148/2320] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:40:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:40:33.222393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=12 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:40:39.087617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=18 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:40:43.965252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:40:44.000827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53395943-d72f-442c-bac3-ea70f0be0872 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:40:44.005899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:40:44.006493 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:40:44.006774 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:40:44.007050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:40:44.016313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:40:44.017953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:40:44.031168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1173/2321] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:40:54.026854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=16 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:43:10.029822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:43:10.066989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b11279-c1be-40ac-8d23-31b6bee775dc service nova] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:43:10.070727 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:43:10.071331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:43:10.071620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:43:10.072146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:43:10.102997 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:43:10.104429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:43:10.116804 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1149/2322] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:43:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:43:11.722888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:43:11.902225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f718f51-135b-44cd-9183-16952ab1da34 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:43:11.908975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153103b-83fd-46b8-bbe0-77b0bb5c0d16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:43:11.909540 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153103b-83fd-46b8-bbe0-77b0bb5c0d16 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:43:11.909949 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153103b-83fd-46b8-bbe0-77b0bb5c0d16 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:43:11.910315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153103b-83fd-46b8-bbe0-77b0bb5c0d16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:43:11.913069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5153103b-83fd-46b8-bbe0-77b0bb5c0d16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:43:11.945221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153103b-83fd-46b8-bbe0-77b0bb5c0d16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:43:11.947219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153103b-83fd-46b8-bbe0-77b0bb5c0d16 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:43:11.960654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1174/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:43:12.574278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:43:12.700837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0c1c0-f9d1-4173-9899-ee9356e48944 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:43:12.704924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:43:12.706868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:43:12.710478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:43:12.710805 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:43:12.727707 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:43:12.729354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:43:12.744148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1150/2324] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:43:15.702411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216f6e6-eb3d-4678-a816-1e47726a01cf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:43:15.702893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216f6e6-eb3d-4678-a816-1e47726a01cf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:43:15.702993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216f6e6-eb3d-4678-a816-1e47726a01cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:43:15.744391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9216f6e6-eb3d-4678-a816-1e47726a01cf None None] MFA Rules not processed for user `a22a06667e10411e8d8ce92db4536a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:43:15.745601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9216f6e6-eb3d-4678-a816-1e47726a01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:43:15.810175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1175/2325] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:43:16.114969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:43:16.123652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06281d-39ac-49c9-a7c7-15e44da02611 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:43:16.127474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:43:16.128060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:43:16.128333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:43:16.128620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:43:16.135748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:43:16.137151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:43:16.144877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1151/2326] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:43:22.213280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216f6e6-eb3d-4678-a816-1e47726a01cf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:43:22.225920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216f6e6-eb3d-4678-a816-1e47726a01cf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:43:22.231194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:43:22.231790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:43:22.232069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:43:22.232334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:43:22.234178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:43:22.244982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:43:22.246607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:43:22.251105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:43:22.260342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1176/2327] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:43:22.310432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:43:22.312030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:43:22.337305 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d48a22-0a8d-409e-a7e9-490eea629254 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:43:22.341049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:43:22.341657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:43:22.341978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:43:22.342252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:43:22.344098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:43:22.370797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:43:22.372222 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:43:22.384106 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1152/2328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:43:32.379241 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=73 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:44:22.311558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:44:22.320955 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504334-eda1-42cb-a84c-2d79e09d8a7d service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:44:22.324530 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:44:22.325115 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:44:22.325393 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:44:22.325665 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:44:22.334199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:44:22.335607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:44:22.362668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1177/2329] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:44:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:44:22.407633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:44:22.417783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1d471-767f-4e52-ade5-c0813efbf0c9 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:44:22.421402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:44:22.422025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:44:22.422304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:44:22.422574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:44:22.431100 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:44:22.432494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:44:22.459580 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1153/2330] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:44:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:44:32.359910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=13 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:44:32.456238 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=13 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:44:40.523890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:44:40.533311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36682e2f-0b78-44b6-8989-425d76a86af1 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:44:40.537553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:44:40.538237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:44:40.538516 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:44:40.538785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:44:40.540631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:44:40.550000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:44:40.551626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:44:40.559367 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1178/2331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:44:50.555910 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:45:10.214145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:45:10.222869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e89b77-bdaf-4498-b681-918d35e5865f service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:45:10.227456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:45:10.228049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:45:10.228331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:45:10.228605 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:45:10.230451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:45:10.309371 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:45:10.310816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:45:10.328939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1154/2332] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:45:20.326443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=41 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:45:48.854600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:45:48.856172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:45:48.935875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d346-6f0a-4412-843b-0edb1f2563da service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:45:48.940851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:45:48.941446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:45:48.941753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:45:48.942022 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:45:48.943823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:45:48.956401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:45:48.958003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:45:48.967988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1179/2333] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:45:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:45:58.963746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=37 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:10.196851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:10.205138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7b368-e5a6-4930-b47c-deb409073a37 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:10.208723 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:10.209310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:10.209585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:10.209893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:10.211669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:10.290467 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:10.292429 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:10.310870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1155/2334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:11.010244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:11.025433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110976c4-0e65-4852-80e2-781978eb00bb service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:11.032830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:11.034738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:11.035478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:11.036127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:11.039179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:11.054619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:11.059318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:11.090638 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1180/2335] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:14.378344 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:14.379998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:14.395107 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cf060-a47c-40ae-a6ee-97752bfb8c1e service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:14.400028 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:14.401444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:14.401688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:14.401995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:14.404569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:14.543287 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:14.544819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:14.573523 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1156/2336] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:15.578322 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:15.589145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0008ee-f9ea-404c-a692-88fbccc3efd7 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:15.593061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:15.593660 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:15.593964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:15.594231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:15.596214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:15.608404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:15.609982 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:15.619809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1181/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:16.211237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:16.222144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe7961-9432-48ab-90f1-c6f7dc8955f7 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:16.226916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:16.227543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:16.227824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:16.228098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:16.230737 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:16.246508 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:16.248197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:16.260009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1157/2338] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:17.605618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:17.620346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df566d7-d573-47c9-9841-0e199871c3c2 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:17.632364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:17.633214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:17.633814 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:17.635501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:17.637991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:17.661526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:17.668368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:17.679659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1182/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:17.756602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:17.758738 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:17.789782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dab4a-5486-4291-9bcb-824104c7ee28 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:17.802728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:17.803460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:17.803748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:17.804029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] PATH_INFO: `/v3/users/3e4eccc7ece24cf4bcfee361d3342a4f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:17.808015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4eccc7ece24cf4bcfee361d3342a4f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:17.814066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:17.845144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:17.910355 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4eccc7ece24cf4bcfee361d3342a4f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:17.932478 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4eccc7ece24cf4bcfee361d3342a4f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:17.967561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4eccc7ece24cf4bcfee361d3342a4f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:17.968696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4eccc7ece24cf4bcfee361d3342a4f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:17.969806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-a482bd89-982e-4dfa-8123-82e7579cf481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4eccc7ece24cf4bcfee361d3342a4f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:17.997438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1158/2340] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:17 2022] DELETE /identity/v3/users/3e4eccc7ece24cf4bcfee361d3342a4f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:18.476908 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:18.508165 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9d916-55f1-4a1a-a27c-c98c1741a145 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:18.632583 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:18.635071 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:18.636898 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:18.637483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] PATH_INFO: `/v3/projects/07cb899fe6934c27820d1329375e044d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:18.639532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07cb899fe6934c27820d1329375e044d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:18.643431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:18.699883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07cb899fe6934c27820d1329375e044d', 'request_id': 'req-a7a5baa1-5293-4242-9955-11f430ed991f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:18.707756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07cb899fe6934c27820d1329375e044d', 'request_id': 'req-a7a5baa1-5293-4242-9955-11f430ed991f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:18.716152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07cb899fe6934c27820d1329375e044d', 'request_id': 'req-a7a5baa1-5293-4242-9955-11f430ed991f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:18.736138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] The token cache is being invalidate because project 07cb899fe6934c27820d1329375e044d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:18.736507 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:18.737508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a7a5baa1-5293-4242-9955-11f430ed991f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:18.739232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1183/2341] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:18 2022] DELETE /identity/v3/projects/07cb899fe6934c27820d1329375e044d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:18.745511 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:18.746040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:18.746326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:18.823039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:18.824995 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:18.904432 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1159/2342] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:18.911961 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a29f-ee82-43bd-86dc-c4c945d95268 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:18.912255 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a29f-ee82-43bd-86dc-c4c945d95268 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:18.912616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a29f-ee82-43bd-86dc-c4c945d95268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:18.933499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:18.935024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-da45a29f-ee82-43bd-86dc-c4c945d95268 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:18.935154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:18.936484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-da45a29f-ee82-43bd-86dc-c4c945d95268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:18.945130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1184/2343] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:18.948302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f954525-444f-4b66-ba3e-9b77deb0f711 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:18.949823 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a29f-ee82-43bd-86dc-c4c945d95268 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:18.954987 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:18.956192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:18.956475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:18.956772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] PATH_INFO: `/v3/users/a22a06667e10411e8d8ce92db4536a1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:18.959555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22a06667e10411e8d8ce92db4536a1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:18.961339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:18.970374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a29f-ee82-43bd-86dc-c4c945d95268 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:18.980282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:18.981110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:18.981574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:18.982029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:18.982860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:18.983335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:18.985427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.004242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1185/2344] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:47:19.008699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.020578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edd4ea-9f81-4479-9d2d-c06fbacbebf6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:19.026424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:19.027279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.027520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.027825 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.028758 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:19.030875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.055055 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22a06667e10411e8d8ce92db4536a1a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.068996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22a06667e10411e8d8ce92db4536a1a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.075036 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=16,keystone:SELECT=83,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:19.083497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:19.095036 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1186/2345] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:19 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:19.099664 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22a06667e10411e8d8ce92db4536a1a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.100105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.100981 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22a06667e10411e8d8ce92db4536a1a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.101702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22a06667e10411e8d8ce92db4536a1a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.122996 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1160/2346] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:18 2022] DELETE /identity/v3/users/a22a06667e10411e8d8ce92db4536a1a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:19.190535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c495d9-2f8c-47f3-a996-77abc5f6f6d1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:19.292242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:19.292868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.293678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.293991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.294699 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:19.296633 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.329772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1187/2347] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:19 2022] POST /identity/v3/users => generated 287 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:19.332716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.334709 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:19.350254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae3955-d1c4-4451-a3a8-253b2b2ae52f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:19.380232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:19.381227 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.381499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.382600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.383689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:19.387266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.405961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1161/2348] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:19.415179 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.433441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25a519-da79-405f-b6d9-adddf4d638a5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:19.441034 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:19.442178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.442651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.443116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.443895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:19.446121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.467640 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.473931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1188/2349] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:19.480791 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fd88-e503-4557-83b0-3c89e42acdc6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:19.486367 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7b6be-e594-4f6e-a922-f29037192611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:19.487331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b6be-e594-4f6e-a922-f29037192611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.487785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b6be-e594-4f6e-a922-f29037192611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.488230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7b6be-e594-4f6e-a922-f29037192611 admin admin] PATH_INFO: `/v3/projects/4c0eff8e63c74120a5e32169e69f12f9/users/abd9c6e1b3af4692888aaa29d06e918f/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.516788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.520307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:19.546830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7b6be-e594-4f6e-a922-f29037192611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0eff8e63c74120a5e32169e69f12f9, user_id=abd9c6e1b3af4692888aaa29d06e918f, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:19.549531 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7b6be-e594-4f6e-a922-f29037192611 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.562589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea36549-2082-4a55-9807-52614ae4f133 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:19.575382 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1162/2350] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:19 2022] PUT /identity/v3/projects/4c0eff8e63c74120a5e32169e69f12f9/users/abd9c6e1b3af4692888aaa29d06e918f/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:19.580500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.581237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.581237 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.615856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:19.618368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:19.694642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:19.695203 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1163/2351] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:19.696011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.696463 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.696740 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] PATH_INFO: `/v3/projects/6136ce7badef4cda8912aa34203a2cc0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.703491 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6136ce7badef4cda8912aa34203a2cc0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:19.705575 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.706999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.709474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:19.778881 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:19.809172 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6e3b0-1601-4fc0-af44-aad9de8281b0 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:19.814475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6136ce7badef4cda8912aa34203a2cc0', 'request_id': 'req-76eed9f1-9568-4153-9734-035131d46d40'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.820803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6136ce7badef4cda8912aa34203a2cc0', 'request_id': 'req-76eed9f1-9568-4153-9734-035131d46d40'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.828153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6136ce7badef4cda8912aa34203a2cc0', 'request_id': 'req-76eed9f1-9568-4153-9734-035131d46d40'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.847089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] The token cache is being invalidate because project 6136ce7badef4cda8912aa34203a2cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:19.847514 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.848459 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76eed9f1-9568-4153-9734-035131d46d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:19.850764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1189/2352] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:19 2022] DELETE /identity/v3/projects/6136ce7badef4cda8912aa34203a2cc0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:19.857011 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.857292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.857569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.879768 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:19.880454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:19.880559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:19.880662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:19.880901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:19.881689 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:19.885282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:19.895515 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:19.958685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1190/2353] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:19.962083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.962083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:19.967098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:19.968555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:19.981321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1164/2354] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:19.982342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:19.984369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.002589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9bc87c-3cff-4ece-9ff3-0285cc237936 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.009244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.010669 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.010952 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.011225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] PATH_INFO: `/v3/users/961012089bd147ecab4fcdf5ea3ae4c3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.012438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961012089bd147ecab4fcdf5ea3ae4c3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.014510 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.040111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93106da7-b977-4f9d-8f9c-0dded1dbe48c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.046865 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4de0b3-b706-4dc4-be35-7d6a4bb41748 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.047531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4de0b3-b706-4dc4-be35-7d6a4bb41748 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.047958 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4de0b3-b706-4dc4-be35-7d6a4bb41748 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.048352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4de0b3-b706-4dc4-be35-7d6a4bb41748 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.051208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e4de0b3-b706-4dc4-be35-7d6a4bb41748 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.068444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4de0b3-b706-4dc4-be35-7d6a4bb41748 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.071951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4de0b3-b706-4dc4-be35-7d6a4bb41748 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.078826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961012089bd147ecab4fcdf5ea3ae4c3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.086400 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961012089bd147ecab4fcdf5ea3ae4c3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.102163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1191/2355] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:20.102163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cb081-92eb-4111-bdd3-9821f98fc11b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.102163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cb081-92eb-4111-bdd3-9821f98fc11b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.102163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cb081-92eb-4111-bdd3-9821f98fc11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.105179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961012089bd147ecab4fcdf5ea3ae4c3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.106053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961012089bd147ecab4fcdf5ea3ae4c3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.106924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961012089bd147ecab4fcdf5ea3ae4c3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.129385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1165/2356] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:19 2022] DELETE /identity/v3/users/961012089bd147ecab4fcdf5ea3ae4c3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:20.141210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c54cb081-92eb-4111-bdd3-9821f98fc11b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:20.142799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c54cb081-92eb-4111-bdd3-9821f98fc11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.193256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:20.213766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1192/2357] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:20.218454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.229359 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6524a51-f11b-4aef-9338-baf57703ac98 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.235557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.236439 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.236845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.237224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.238881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:20.239901 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.242110 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.258116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1166/2358] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:47:20.262936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cb081-92eb-4111-bdd3-9821f98fc11b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.272609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cb081-92eb-4111-bdd3-9821f98fc11b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.277998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.278616 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.278897 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.279172 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.280177 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.281755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.315484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1193/2359] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:20.319863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.330621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd697b5-a93c-4c86-8f5d-8201b111c755 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.373460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.375528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.412565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.415005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.415197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.415747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.417386 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.419748 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.434893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:20.453821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1167/2360] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:20.455139 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.460561 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c90b4-959b-4397-b4bb-86556cdaa714 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.467652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.468301 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.468581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.468856 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] PATH_INFO: `/v3/projects/4a11c6709cba4247acc57bd97250f8a0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.469302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f99c0-e3fe-44f7-b031-782d737ca2eb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.470157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a11c6709cba4247acc57bd97250f8a0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.472206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.475029 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.475957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.476302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.476773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.477487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.480419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.494232 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1168/2361] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:20.498108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.512240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc894b3-297c-4e0c-aa0f-9c0f49ff16d9 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.520587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.521335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.522093 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.522450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.523211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.526092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.542667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1169/2362] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:20.542847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a11c6709cba4247acc57bd97250f8a0', 'request_id': 'req-08f4d2d6-b295-4650-8676-0bb5717401fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.544745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.549407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a11c6709cba4247acc57bd97250f8a0', 'request_id': 'req-08f4d2d6-b295-4650-8676-0bb5717401fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.563122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946c300-3a0a-46d8-8616-fa78bd16288e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.563441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a11c6709cba4247acc57bd97250f8a0', 'request_id': 'req-08f4d2d6-b295-4650-8676-0bb5717401fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.577280 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] The token cache is being invalidate because project 4a11c6709cba4247acc57bd97250f8a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:20.577563 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.578391 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-08f4d2d6-b295-4650-8676-0bb5717401fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:20.580366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1194/2363] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:20 2022] DELETE /identity/v3/projects/4a11c6709cba4247acc57bd97250f8a0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:20.585988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb6a71-8cf3-46ac-97ce-10fad6d9d857 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.586259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb6a71-8cf3-46ac-97ce-10fad6d9d857 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.586560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb6a71-8cf3-46ac-97ce-10fad6d9d857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.607499 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6bfb6a71-8cf3-46ac-97ce-10fad6d9d857 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:20.608704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6bfb6a71-8cf3-46ac-97ce-10fad6d9d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.635702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f45f70-b6b6-45ff-8f5e-ef824606d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.637116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f45f70-b6b6-45ff-8f5e-ef824606d8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.638083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f45f70-b6b6-45ff-8f5e-ef824606d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.639304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f45f70-b6b6-45ff-8f5e-ef824606d8d6 admin admin] PATH_INFO: `/v3/projects/87800823703e407f9ec8380193298d83/users/92de7114cd8347b39d61cad86dcbfebe/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.656979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f45f70-b6b6-45ff-8f5e-ef824606d8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87800823703e407f9ec8380193298d83, user_id=92de7114cd8347b39d61cad86dcbfebe, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.659687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f45f70-b6b6-45ff-8f5e-ef824606d8d6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.664090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:20.676066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1170/2364] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:20 2022] PUT /identity/v3/projects/87800823703e407f9ec8380193298d83/users/92de7114cd8347b39d61cad86dcbfebe/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:20.690300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.690300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.690300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.690670 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1195/2365] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:20.691966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ce9ea-f7a9-4ade-bcd5-64c2d939bc64 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.692258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ce9ea-f7a9-4ade-bcd5-64c2d939bc64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.692581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ce9ea-f7a9-4ade-bcd5-64c2d939bc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.714267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3b9ce9ea-f7a9-4ade-bcd5-64c2d939bc64 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:20.715593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3b9ce9ea-f7a9-4ade-bcd5-64c2d939bc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.721920 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] MFA Rules not processed for user `92de7114cd8347b39d61cad86dcbfebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:20.723391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.804573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1196/2366] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:20.809191 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ce9ea-f7a9-4ade-bcd5-64c2d939bc64 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.815472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1171/2367] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:20.819794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ce9ea-f7a9-4ade-bcd5-64c2d939bc64 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.819958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.821190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:20.825588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.826232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.826511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.826783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.827529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:20.827727 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.829414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.844828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1197/2368] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:47:20.849649 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.852877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901ba6b-f5d1-4359-9ecb-d4d23e8e3939 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.860774 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.862042 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.862436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.862716 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.863506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.863713 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04158a9f-7b59-42c0-8d7b-c6eaa35bbf0b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:20.866547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.869942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:20.870782 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:20.871219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:20.871651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:20.872474 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:20.874839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:20.907689 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1172/2369] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:20.908760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.914111 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1198/2370] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:20.922755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:20.949385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:20.999239 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fef40c-db3e-486f-82d1-b2fde8cb2f96 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.005372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e60f50-ccab-4a2c-b011-8a9cdcb40f1a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.005524 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.006157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.006438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.006700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.007460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.009450 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.011877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.012492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.012775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.013048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.013688 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.015794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.038056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1199/2371] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:21.043330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.050506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1173/2372] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:20 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:21.054306 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.057712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b9b34-381d-47f8-bc04-cf3bc6867a2f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.062913 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.063857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.064282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.064710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.065573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.066104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fbc41-4a90-4989-bd95-347bf0b7df45 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.067998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.073494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.075030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.075451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.075822 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.076678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.078966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.085866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1200/2373] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:21.088831 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.095911 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1174/2374] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:21.097569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.114096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e24f-6667-49d0-b220-1e47e4b5b6ad admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.117695 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bfb70-510e-431f-b1f8-8702490e95b9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.119998 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.120811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.121193 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.121633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] PATH_INFO: `/v3/projects/23c5226266d24e03af1bf61b239a2509/users/2c9204d6a45e407191369b0dabf61583/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.125163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.126020 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.126308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.126588 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.127243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.129764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.140983 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c5226266d24e03af1bf61b239a2509, user_id=2c9204d6a45e407191369b0dabf61583, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.143319 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1201/2375] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:21.147989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.148133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.167828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d982ca1-5336-44b4-9a40-d6117152d098 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.173813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1175/2376] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:21 2022] PUT /identity/v3/projects/23c5226266d24e03af1bf61b239a2509/users/2c9204d6a45e407191369b0dabf61583/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:21.174826 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.205360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:21.263312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6d7af-ddaf-4550-9d80-caf23336ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.263985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6d7af-ddaf-4550-9d80-caf23336ed8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.264268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6d7af-ddaf-4550-9d80-caf23336ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.264584 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6d7af-ddaf-4550-9d80-caf23336ed8a admin admin] PATH_INFO: `/v3/projects/aaee679d12b94bcab792ffdd0f2ec333/users/2e27ac5355154d5a810d4a1da0416dc5/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.269312 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53adaa-951b-41e6-ae8f-4507a01ebd0e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.278389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.279464 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.280026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.280612 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.281424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.284251 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.284342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6d7af-ddaf-4550-9d80-caf23336ed8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaee679d12b94bcab792ffdd0f2ec333, user_id=2e27ac5355154d5a810d4a1da0416dc5, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.287787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6d7af-ddaf-4550-9d80-caf23336ed8a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.301318 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1176/2377] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:21.302634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.318600 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c727-35a8-494b-aed3-075ec3554638 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.318829 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1202/2378] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:21 2022] PUT /identity/v3/projects/aaee679d12b94bcab792ffdd0f2ec333/users/2e27ac5355154d5a810d4a1da0416dc5/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:21.323447 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.323918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.324341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.384235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] MFA Rules not processed for user `2e27ac5355154d5a810d4a1da0416dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:21.387395 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:21.402134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827cfec7-6b7c-4bea-ae13-bc6656aeeb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.403170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cfec7-6b7c-4bea-ae13-bc6656aeeb9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.403585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cfec7-6b7c-4bea-ae13-bc6656aeeb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.404038 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827cfec7-6b7c-4bea-ae13-bc6656aeeb9a admin admin] PATH_INFO: `/v3/projects/23c5226266d24e03af1bf61b239a2509/users/2c9204d6a45e407191369b0dabf61583/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.406645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cfec7-6b7c-4bea-ae13-bc6656aeeb9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c5226266d24e03af1bf61b239a2509, user_id=2c9204d6a45e407191369b0dabf61583, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.410368 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827cfec7-6b7c-4bea-ae13-bc6656aeeb9a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.449268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1177/2379] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:21 2022] PUT /identity/v3/projects/23c5226266d24e03af1bf61b239a2509/users/2c9204d6a45e407191369b0dabf61583/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:21.451484 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.451926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.452430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.483518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:21.497771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] MFA Rules not processed for user `2c9204d6a45e407191369b0dabf61583`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:21.499231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:21.512804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1203/2380] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:21.543214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.545247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:21.608994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1178/2381] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:21.623032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:21.624868 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:21.647935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5eb38a-2544-4486-b893-7d31b0f5d74e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.707168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:21.722675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac8627-3e33-4d94-882d-f230fa61944a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:21.730178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.732269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.732559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.732835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.744752 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.746457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.749720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a81b2-ee13-40a3-8887-b2a474d0a61c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:21.750383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a81b2-ee13-40a3-8887-b2a474d0a61c service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:21.750783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a81b2-ee13-40a3-8887-b2a474d0a61c service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:21.751236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a81b2-ee13-40a3-8887-b2a474d0a61c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:21.759231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1179/2382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:47:21.856031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a81b2-ee13-40a3-8887-b2a474d0a61c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:21.858176 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a81b2-ee13-40a3-8887-b2a474d0a61c service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:21.895031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1204/2383] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:22.410202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:22.487869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4ab7a-5ad1-45b2-8bf8-28dd78d6fc16 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:22.499773 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:22.500860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:22.501145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:22.501420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] PATH_INFO: `/v3/users/92de7114cd8347b39d61cad86dcbfebe` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:22.502654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] RBAC: Authorizing `identity:delete_user(user_id=92de7114cd8347b39d61cad86dcbfebe)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:22.504571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:22.566959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:22.575254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92de7114cd8347b39d61cad86dcbfebe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:22.585692 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92de7114cd8347b39d61cad86dcbfebe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:22.602982 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92de7114cd8347b39d61cad86dcbfebe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:22.604507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92de7114cd8347b39d61cad86dcbfebe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:22.605188 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92de7114cd8347b39d61cad86dcbfebe'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:22.621979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1180/2384] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:22 2022] DELETE /identity/v3/users/92de7114cd8347b39d61cad86dcbfebe => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:22.626128 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:22.626632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:22.626980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:22.648937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:22.651141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:22.721850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1205/2385] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:22.732132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:22.733426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:22.800410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b03ab-7027-46e3-867c-c3558730d41b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:22.804502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:22.805147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:22.805423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:22.805870 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:22.817843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:22.819282 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:22.833902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1181/2386] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:23.257637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:23.259330 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:23.280315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fc892-c9b5-4cb8-97ef-2b05ef4ae8a1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:23.308651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:23.309620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:23.310777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:23.311302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] PATH_INFO: `/v3/projects/87800823703e407f9ec8380193298d83` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:23.313121 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87800823703e407f9ec8380193298d83)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:23.316375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:23.400795 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87800823703e407f9ec8380193298d83', 'request_id': 'req-d714b88e-47b0-4c76-b543-ee040307f004'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.430550 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87800823703e407f9ec8380193298d83', 'request_id': 'req-d714b88e-47b0-4c76-b543-ee040307f004'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.447803 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87800823703e407f9ec8380193298d83', 'request_id': 'req-d714b88e-47b0-4c76-b543-ee040307f004'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.499675 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:23.517080 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] The token cache is being invalidate because project 87800823703e407f9ec8380193298d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:23.521226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.522442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.525512 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1206/2387] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:23 2022] DELETE /identity/v3/projects/87800823703e407f9ec8380193298d83 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:23.535150 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:23.556133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7d5c2-61ec-4118-b047-876779f2b1e0 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:23.698988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:23.699641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:23.699925 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:23.700207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] PATH_INFO: `/v3/users/2c9204d6a45e407191369b0dabf61583` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:23.701378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c9204d6a45e407191369b0dabf61583)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:23.703678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:23.778104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c9204d6a45e407191369b0dabf61583'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.787794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c9204d6a45e407191369b0dabf61583'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.794387 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:23.818802 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c9204d6a45e407191369b0dabf61583'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.820395 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c9204d6a45e407191369b0dabf61583'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.821256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-7b07a175-1487-4843-a371-374a0587e040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c9204d6a45e407191369b0dabf61583'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:23.844343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1182/2388] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:23 2022] DELETE /identity/v3/users/2c9204d6a45e407191369b0dabf61583 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:24.220669 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:24.224488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:24.261478 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b88e-47b0-4c76-b543-ee040307f004 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:24.356604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:24.357776 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.358161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.358527 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] PATH_INFO: `/v3/projects/23c5226266d24e03af1bf61b239a2509` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.361444 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c5226266d24e03af1bf61b239a2509)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:24.363642 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:24.438522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c5226266d24e03af1bf61b239a2509', 'request_id': 'req-85bb0f7f-7f1e-49cb-8d14-466c817fe823'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:24.445706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c5226266d24e03af1bf61b239a2509', 'request_id': 'req-85bb0f7f-7f1e-49cb-8d14-466c817fe823'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:24.455211 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c5226266d24e03af1bf61b239a2509', 'request_id': 'req-85bb0f7f-7f1e-49cb-8d14-466c817fe823'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:24.474593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] The token cache is being invalidate because project 23c5226266d24e03af1bf61b239a2509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:24.474925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:24.476446 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-85bb0f7f-7f1e-49cb-8d14-466c817fe823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:24.478998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1207/2389] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:24 2022] DELETE /identity/v3/projects/23c5226266d24e03af1bf61b239a2509 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:24.483667 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508df50-0f66-46e8-8bfd-861e23fcb4a7 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.483867 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508df50-0f66-46e8-8bfd-861e23fcb4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.484054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508df50-0f66-46e8-8bfd-861e23fcb4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.534015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-7508df50-0f66-46e8-8bfd-861e23fcb4a7 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:24.535251 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-7508df50-0f66-46e8-8bfd-861e23fcb4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:24.617228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1183/2390] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:24.626376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ea57-9912-48a7-bebf-c27077cbd26a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.626813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ea57-9912-48a7-bebf-c27077cbd26a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.627088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099ea57-9912-48a7-bebf-c27077cbd26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.649407 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e099ea57-9912-48a7-bebf-c27077cbd26a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:24.651473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e099ea57-9912-48a7-bebf-c27077cbd26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:24.661324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1208/2391] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:24.666345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508df50-0f66-46e8-8bfd-861e23fcb4a7 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:24.677700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508df50-0f66-46e8-8bfd-861e23fcb4a7 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:24.682611 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:24.683225 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.683506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.683785 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.684449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:24.684806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:24.686430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:24.699971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1184/2392] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:47:24.704363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099ea57-9912-48a7-bebf-c27077cbd26a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:24.719046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099ea57-9912-48a7-bebf-c27077cbd26a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:24.723723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:24.724327 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.724661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.724878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.725509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:24.727445 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:24.761980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1209/2393] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:24 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:24.763585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:24.777013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2fcc3-a550-4fe0-a2fa-84d6585c319d admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:24.865212 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:24.865904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.868546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.868546 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.870059 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:24.872143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:24.909789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1185/2394] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:24 2022] POST /identity/v3/users => generated 282 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:24.911296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:24.922912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c71fe1-3499-40b1-a57c-9a7f1cad2f99 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:24.928313 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:24.929008 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.929455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.929862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.930640 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:24.932780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:24.944195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1210/2395] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:24.948847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:24.968871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f81c6-27cf-4365-a93b-df775cd88a47 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:24.975334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840d2ef-94d0-40b1-994b-17c3217f8806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:24.975947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840d2ef-94d0-40b1-994b-17c3217f8806 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:24.976170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840d2ef-94d0-40b1-994b-17c3217f8806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:24.976544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840d2ef-94d0-40b1-994b-17c3217f8806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:24.977947 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840d2ef-94d0-40b1-994b-17c3217f8806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:24.980057 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840d2ef-94d0-40b1-994b-17c3217f8806 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:24.994404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1186/2396] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:25.001752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.014018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0362bd-1418-4701-9d40-a04b4aa46e6d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.019739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.020704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.021126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.021918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] PATH_INFO: `/v3/projects/811482ba1a6846b585f0a6b4fac3d9c3/users/fecb9bd79a6e4f7480f0db0f6f6528ef/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.039641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=811482ba1a6846b585f0a6b4fac3d9c3, user_id=fecb9bd79a6e4f7480f0db0f6f6528ef, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.044268 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.089847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1211/2397] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:24 2022] PUT /identity/v3/projects/811482ba1a6846b585f0a6b4fac3d9c3/users/fecb9bd79a6e4f7480f0db0f6f6528ef/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:25.093131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef0aa-14ee-48dc-b257-2f2ae99d0f85 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.093527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef0aa-14ee-48dc-b257-2f2ae99d0f85 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.094046 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef0aa-14ee-48dc-b257-2f2ae99d0f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.133466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ac8ef0aa-14ee-48dc-b257-2f2ae99d0f85 None None] MFA Rules not processed for user `fecb9bd79a6e4f7480f0db0f6f6528ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:25.135384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ac8ef0aa-14ee-48dc-b257-2f2ae99d0f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:25.175524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:25.219885 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1187/2398] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:25.224602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.246414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.247064 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.247651 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.281554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] MFA Rules not processed for user `abd9c6e1b3af4692888aaa29d06e918f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:25.283133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:25.283745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:25.340783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea4b9c-a809-4997-9c7e-9f9fdcdaabaf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.345748 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.346442 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.346808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.347141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.347921 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.349859 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.368114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1188/2399] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:25.373460 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.375218 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:25.384760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1212/2400] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:25 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:25.388899 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.391226 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160a1de-a4a2-4a2f-8910-b256f7cfc206 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.490567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e8ef3-1255-4b57-975c-5db5b5774c62 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.502732 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.503355 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.503588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.503977 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.504088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.504254 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.504517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.504730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.505361 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.505496 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.507532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.509056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.545574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1213/2401] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:25 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:25.547517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1189/2402] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:25 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:25.549232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.552569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.564209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fb021-9cd9-4d27-9101-9369950eacda admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.642017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:25.658942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ea3fb-d420-4a5b-93aa-0f0f59a2867a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.664231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.665157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.665271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.666407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.667311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.669916 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.672495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.673110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.673398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.673671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.674361 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.675963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.694659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1214/2403] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:25.695808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.697934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1190/2404] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:25 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:25.699236 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.707629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7022fe-a45e-466d-a8b9-cc93300a1e97 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.710823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566fa35-b04b-4aba-88fb-4c7a29ed82d8 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.714912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.715579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.715867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.716133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] PATH_INFO: `/v3/projects/c0b2081df01a4d4682c6fe028bcde02d/users/313f3ed12d6440418b0769d0a48146b5/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.716887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.717563 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.717955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.718310 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.719037 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.721271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.733450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1191/2405] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:25.736357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b2081df01a4d4682c6fe028bcde02d, user_id=313f3ed12d6440418b0769d0a48146b5, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.737898 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.739215 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.749445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431afaf-b7a4-47b6-a23b-55ae5b13a367 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.753644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1215/2406] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:25 2022] PUT /identity/v3/projects/c0b2081df01a4d4682c6fe028bcde02d/users/313f3ed12d6440418b0769d0a48146b5/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:25.757605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.777154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:25.815780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.816447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.816725 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.817085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] PATH_INFO: `/v3/projects/1089f8691dd44b399863053f11cf60ab/users/29382da82d8946b1bc871b9a77e1e1f8/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.832607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191119ad-386f-4aaf-8a64-ec9345a16b8f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.833431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1089f8691dd44b399863053f11cf60ab, user_id=29382da82d8946b1bc871b9a77e1e1f8, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.836296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.838270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.839154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.839654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.839953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.841158 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.843010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.853561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1192/2407] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:25 2022] PUT /identity/v3/projects/1089f8691dd44b399863053f11cf60ab/users/29382da82d8946b1bc871b9a77e1e1f8/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:25.855312 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1216/2408] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:25.857384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.859038 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.868737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2c769-9826-4933-94e4-198ddab32195 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.920372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647de42b-e1a4-4bcc-99ac-ddc2ea3290e2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.925157 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.925818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.926101 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.926373 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.927016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.928638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.928920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef8561-6543-4874-9d3e-7450ae3385bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:25.929567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef8561-6543-4874-9d3e-7450ae3385bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.929878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef8561-6543-4874-9d3e-7450ae3385bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.930146 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef8561-6543-4874-9d3e-7450ae3385bf admin admin] PATH_INFO: `/v3/projects/c0b2081df01a4d4682c6fe028bcde02d/users/313f3ed12d6440418b0769d0a48146b5/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.932847 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef8561-6543-4874-9d3e-7450ae3385bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b2081df01a4d4682c6fe028bcde02d, user_id=313f3ed12d6440418b0769d0a48146b5, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:25.935697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef8561-6543-4874-9d3e-7450ae3385bf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:25.939500 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1193/2409] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:25.943492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:25.948069 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1217/2410] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:25 2022] PUT /identity/v3/projects/c0b2081df01a4d4682c6fe028bcde02d/users/313f3ed12d6440418b0769d0a48146b5/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:25.954353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ac888-4710-4da5-8645-29736a5dbecb admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:25.959368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6982c9-d2e8-45c0-9eec-e1ab4b111bf2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:25.959655 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6982c9-d2e8-45c0-9eec-e1ab4b111bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:25.959925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6982c9-d2e8-45c0-9eec-e1ab4b111bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:25.990619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0b6982c9-d2e8-45c0-9eec-e1ab4b111bf2 None None] MFA Rules not processed for user `313f3ed12d6440418b0769d0a48146b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:25.991783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b6982c9-d2e8-45c0-9eec-e1ab4b111bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:26.003846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:26.019559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:26.020252 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:26.020526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:26.020800 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] PATH_INFO: `/v3/projects/1089f8691dd44b399863053f11cf60ab/users/29382da82d8946b1bc871b9a77e1e1f8/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:26.023786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1089f8691dd44b399863053f11cf60ab, user_id=29382da82d8946b1bc871b9a77e1e1f8, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:26.027470 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:26.040300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1194/2411] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:25 2022] PUT /identity/v3/projects/1089f8691dd44b399863053f11cf60ab/users/29382da82d8946b1bc871b9a77e1e1f8/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:26.073645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1218/2412] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:26.085060 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:26.087161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:26.177862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3028b-2fc6-41fb-8602-af436418718b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:26.182697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:26.183449 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:26.183779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:26.184253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:26.260447 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:26.261969 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:26.273594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1195/2413] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:26.355830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6982c9-d2e8-45c0-9eec-e1ab4b111bf2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:26.395484 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:26.428611 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6982c9-d2e8-45c0-9eec-e1ab4b111bf2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:26.433452 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e914f4-fe9f-43fb-87f0-05e22f7c5010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:26.434418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e914f4-fe9f-43fb-87f0-05e22f7c5010 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:26.434802 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e914f4-fe9f-43fb-87f0-05e22f7c5010 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:26.435154 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e914f4-fe9f-43fb-87f0-05e22f7c5010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:26.514653 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e914f4-fe9f-43fb-87f0-05e22f7c5010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:26.516890 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e914f4-fe9f-43fb-87f0-05e22f7c5010 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:26.550349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1219/2414] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:30.126790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:30.128964 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:30.356953 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bef9d4-ca40-4172-a0f1-13777e5717e7 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:30.373802 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:30.374472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:30.374755 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:30.375030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] PATH_INFO: `/v3/users/2e27ac5355154d5a810d4a1da0416dc5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:30.377660 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e27ac5355154d5a810d4a1da0416dc5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:30.379638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:30.499440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:30.522989 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e27ac5355154d5a810d4a1da0416dc5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:30.553857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e27ac5355154d5a810d4a1da0416dc5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:30.584465 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e27ac5355154d5a810d4a1da0416dc5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:30.585231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e27ac5355154d5a810d4a1da0416dc5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:30.585965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e27ac5355154d5a810d4a1da0416dc5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:30.615499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1196/2415] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:30 2022] DELETE /identity/v3/users/2e27ac5355154d5a810d4a1da0416dc5 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:30.620591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c4b3-d9fc-41a0-b1b9-00a3c738becd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:30.621100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c4b3-d9fc-41a0-b1b9-00a3c738becd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:30.621361 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4c4b3-d9fc-41a0-b1b9-00a3c738becd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:30.679718 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bfc4c4b3-d9fc-41a0-b1b9-00a3c738becd None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:30.681151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bfc4c4b3-d9fc-41a0-b1b9-00a3c738becd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:30.830978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1220/2416] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:30.847124 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:30.849517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:30.967197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e6fa8-a323-4700-8f81-87ad65568e7c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:30.970931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:30.971559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:30.971844 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:30.972116 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:30.998020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:30.999472 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:31.015782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1197/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:31.060756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.061342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.061626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.130791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] MFA Rules not processed for user `29382da82d8946b1bc871b9a77e1e1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:31.132017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:31.251592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1221/2418] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:31.268020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:31.269795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:31.357787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac1ad3-b2b4-477a-ae2c-47dcbc99e50f service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:31.366175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:31.368302 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:31.379875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ef06-423b-4795-b4d6-d9916fef7204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:31.380520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ef06-423b-4795-b4d6-d9916fef7204 service nova] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.380806 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ef06-423b-4795-b4d6-d9916fef7204 service nova] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.381086 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ef06-423b-4795-b4d6-d9916fef7204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.386813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5552ef06-423b-4795-b4d6-d9916fef7204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:31.392161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80649c-c967-412f-9e23-ef1fc39bb1f7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:31.404636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:31.405288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.405609 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.405853 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] PATH_INFO: `/v3/projects/aaee679d12b94bcab792ffdd0f2ec333` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.406987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaee679d12b94bcab792ffdd0f2ec333)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:31.408888 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:31.413124 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:31.456301 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:31.472054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ef06-423b-4795-b4d6-d9916fef7204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:31.474228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ef06-423b-4795-b4d6-d9916fef7204 service nova] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:31.485014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaee679d12b94bcab792ffdd0f2ec333', 'request_id': 'req-55a1bbf5-6910-4377-bdd4-0488eecac1a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:31.492091 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1198/2419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:31.498153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaee679d12b94bcab792ffdd0f2ec333', 'request_id': 'req-55a1bbf5-6910-4377-bdd4-0488eecac1a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:31.505110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaee679d12b94bcab792ffdd0f2ec333', 'request_id': 'req-55a1bbf5-6910-4377-bdd4-0488eecac1a8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:31.523285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] The token cache is being invalidate because project aaee679d12b94bcab792ffdd0f2ec333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:31.523579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:31.524248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-55a1bbf5-6910-4377-bdd4-0488eecac1a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:31.527221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1222/2420] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:31 2022] DELETE /identity/v3/projects/aaee679d12b94bcab792ffdd0f2ec333 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:31.532148 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693e4a5-4af2-4eec-918b-723c9cd9642f None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.532772 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693e4a5-4af2-4eec-918b-723c9cd9642f None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.533167 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693e4a5-4af2-4eec-918b-723c9cd9642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.562349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8693e4a5-4af2-4eec-918b-723c9cd9642f None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:31.563652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8693e4a5-4af2-4eec-918b-723c9cd9642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:31.639396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1199/2421] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:31.646943 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e415a-19b2-413c-91ab-c416d2438fb0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.647738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e415a-19b2-413c-91ab-c416d2438fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.648236 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e415a-19b2-413c-91ab-c416d2438fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.678427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-625e415a-19b2-413c-91ab-c416d2438fb0 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:31.679996 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-625e415a-19b2-413c-91ab-c416d2438fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:31.688971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1223/2422] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:31.692991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693e4a5-4af2-4eec-918b-723c9cd9642f None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:31.705543 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693e4a5-4af2-4eec-918b-723c9cd9642f None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:31.710713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:31.711338 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.711675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.712201 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.713775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:31.714144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:31.716041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:31.731013 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1200/2423] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:47:31.735298 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e415a-19b2-413c-91ab-c416d2438fb0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:31.749811 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e415a-19b2-413c-91ab-c416d2438fb0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:31.764414 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:31.765697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.766951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.766951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.770586 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:31.773872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:31.829141 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1224/2424] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:31 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:31.833048 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:31.844030 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b0b4c-e4c9-4f26-9167-cd856c7f64ac admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:31.923118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:31.923789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:31.924173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:31.924549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:31.925307 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:31.927379 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:31.981592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1201/2425] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:31 2022] POST /identity/v3/users => generated 282 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:31.986083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.015532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f988c-f7fa-4dbe-9317-e9bfc96d4a38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.027054 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.028090 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.028559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.028903 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.030209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.032157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.046844 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1225/2426] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:32.048334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.062175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fdf8e-aa33-49e8-99ff-65766ec6152b admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.068336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0a1fc-d7d8-44df-a0fb-2be95a57b2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.068951 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a1fc-d7d8-44df-a0fb-2be95a57b2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.069567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a1fc-d7d8-44df-a0fb-2be95a57b2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.069567 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a1fc-d7d8-44df-a0fb-2be95a57b2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.070154 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0a1fc-d7d8-44df-a0fb-2be95a57b2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.072369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0a1fc-d7d8-44df-a0fb-2be95a57b2c6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.093322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1202/2427] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:32.095097 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.110100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c29cf-0030-4a93-9eaa-da8ac77acec3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.115178 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.115962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.116651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.116651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] PATH_INFO: `/v3/projects/2379a5365b7645f68ae12dc5feeb64eb/users/a357abd4a2c2436ba985d9d3e8534d59/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.138360 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2379a5365b7645f68ae12dc5feeb64eb, user_id=a357abd4a2c2436ba985d9d3e8534d59, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.142375 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.160987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1226/2428] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:32 2022] PUT /identity/v3/projects/2379a5365b7645f68ae12dc5feeb64eb/users/a357abd4a2c2436ba985d9d3e8534d59/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:32.172923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cf5ae-41e2-4e52-af17-3d63b028c97b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.173277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cf5ae-41e2-4e52-af17-3d63b028c97b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.173551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cf5ae-41e2-4e52-af17-3d63b028c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.206841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-945cf5ae-41e2-4e52-af17-3d63b028c97b None None] MFA Rules not processed for user `a357abd4a2c2436ba985d9d3e8534d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:32.208507 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-945cf5ae-41e2-4e52-af17-3d63b028c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:32.264677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:32.288163 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1203/2429] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:32.294866 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.361933 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34fa17-02c2-48d0-b230-3aefa4b882b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.366766 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.367376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.367662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.367969 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.368635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.370594 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.405857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1227/2430] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:32 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:32.407003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cf5ae-41e2-4e52-af17-3d63b028c97b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.420399 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cf5ae-41e2-4e52-af17-3d63b028c97b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.487228 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.488071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.488466 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.488824 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.489942 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.492278 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.516631 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1204/2431] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:32 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:32.520565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.535592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a31fa-e8cf-43fc-be3d-097f2abb9121 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.540967 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.541704 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.542475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.542475 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.543051 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.544761 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.558693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1228/2432] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:32.562930 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.575656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b6438-b17b-4e32-bddd-e5e178e09b60 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.580562 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ae9b6-1f80-4484-9b94-f8fc5809a9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.581434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae9b6-1f80-4484-9b94-f8fc5809a9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.581766 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae9b6-1f80-4484-9b94-f8fc5809a9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.582093 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ae9b6-1f80-4484-9b94-f8fc5809a9f4 admin admin] PATH_INFO: `/v3/projects/479850a7717944929bfd825d3fc8a701/users/b14941613dd9429badcfe17095ba5cfe/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.600040 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ae9b6-1f80-4484-9b94-f8fc5809a9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479850a7717944929bfd825d3fc8a701, user_id=b14941613dd9429badcfe17095ba5cfe, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.602813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ae9b6-1f80-4484-9b94-f8fc5809a9f4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.616607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1205/2433] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:32 2022] PUT /identity/v3/projects/479850a7717944929bfd825d3fc8a701/users/b14941613dd9429badcfe17095ba5cfe/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:32.620723 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.629965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:32.636860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d578e4e-079e-4fff-babf-96e27ec369f2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.711333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.711946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.712240 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.712565 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.713618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.715581 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.724858 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.725245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.725604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.733679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1229/2434] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:32.737712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.748599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1145fb-1e56-41db-b45d-6be3fa5fb503 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:32.749811 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] MFA Rules not processed for user `abd9c6e1b3af4692888aaa29d06e918f`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:32.751070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:32.753295 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744f60d-d53b-48c5-b646-406385a185f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:32.753979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f60d-d53b-48c5-b646-406385a185f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.754266 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f60d-d53b-48c5-b646-406385a185f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.754569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f60d-d53b-48c5-b646-406385a185f5 admin admin] PATH_INFO: `/v3/projects/479850a7717944929bfd825d3fc8a701/users/b14941613dd9429badcfe17095ba5cfe/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.756498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744f60d-d53b-48c5-b646-406385a185f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479850a7717944929bfd825d3fc8a701, user_id=b14941613dd9429badcfe17095ba5cfe, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:32.759263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744f60d-d53b-48c5-b646-406385a185f5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:32.769646 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1230/2435] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:32 2022] PUT /identity/v3/projects/479850a7717944929bfd825d3fc8a701/users/b14941613dd9429badcfe17095ba5cfe/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:32.785827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eedaa-3ad3-499d-bedc-f0a06b6dbed4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:32.786163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eedaa-3ad3-499d-bedc-f0a06b6dbed4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:32.786449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eedaa-3ad3-499d-bedc-f0a06b6dbed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:32.832153 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-866eedaa-3ad3-499d-bedc-f0a06b6dbed4 None None] MFA Rules not processed for user `b14941613dd9429badcfe17095ba5cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:32.833481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-866eedaa-3ad3-499d-bedc-f0a06b6dbed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:32.847736 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1206/2436] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:32.926500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1231/2437] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:32.938235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:32.939633 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:33.007661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f6d5a-1661-4398-8d12-e0ad343dbc22 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:33.018033 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:33.066532 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:33.067436 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:33.067719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:33.068066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:33.083267 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:33.084873 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:33.096914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1207/2438] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:33.285717 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eedaa-3ad3-499d-bedc-f0a06b6dbed4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:33.297009 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eedaa-3ad3-499d-bedc-f0a06b6dbed4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:33.317817 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc02ea-cb7c-402d-8c2c-aeafd3467c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:33.318674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc02ea-cb7c-402d-8c2c-aeafd3467c81 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:33.318954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc02ea-cb7c-402d-8c2c-aeafd3467c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:33.319234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc02ea-cb7c-402d-8c2c-aeafd3467c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:33.414770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc02ea-cb7c-402d-8c2c-aeafd3467c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:33.417476 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc02ea-cb7c-402d-8c2c-aeafd3467c81 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:33.430129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1232/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:36.218700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:36.281066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b380a2-ee76-41c3-a41e-f6f2655a6ca7 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:36.285834 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:36.286547 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:36.286730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:36.287003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] PATH_INFO: `/v3/users/a357abd4a2c2436ba985d9d3e8534d59` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:36.288271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a357abd4a2c2436ba985d9d3e8534d59)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:36.290214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:36.336780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a357abd4a2c2436ba985d9d3e8534d59'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.344533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a357abd4a2c2436ba985d9d3e8534d59'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.357343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a357abd4a2c2436ba985d9d3e8534d59'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.358075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a357abd4a2c2436ba985d9d3e8534d59'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.358697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a357abd4a2c2436ba985d9d3e8534d59'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.381095 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1208/2440] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:36 2022] DELETE /identity/v3/users/a357abd4a2c2436ba985d9d3e8534d59 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:36.387347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:36.387909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:36.388021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:36.408434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:36.409644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:36.413781 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:36.472014 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1233/2441] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:36.482554 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:36.484178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:36.524702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:36.549196 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb713ec-e04a-443a-a7d2-71fe0c06a7c1 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:36.552905 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:36.553519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:36.553836 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:36.554114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:36.564216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:36.565643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:36.576017 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1209/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:36.866874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:36.868376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:36.893918 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057caae-42c0-4a7b-897c-ba8b33a4492e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:36.901958 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:36.902606 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:36.902889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:36.903161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] PATH_INFO: `/v3/projects/2379a5365b7645f68ae12dc5feeb64eb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:36.904307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2379a5365b7645f68ae12dc5feeb64eb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:36.906277 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:36.927063 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:36.929027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:36.944231 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac88b2-cd1e-4de7-9098-29fb2f715401 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:36.950542 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:36.951280 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:36.951623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:36.951962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] PATH_INFO: `/v3/users/fecb9bd79a6e4f7480f0db0f6f6528ef` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:36.953155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fecb9bd79a6e4f7480f0db0f6f6528ef)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:36.955314 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:36.965873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2379a5365b7645f68ae12dc5feeb64eb', 'request_id': 'req-35782e1b-400d-47b0-9af6-4c2ab59226fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.972644 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2379a5365b7645f68ae12dc5feeb64eb', 'request_id': 'req-35782e1b-400d-47b0-9af6-4c2ab59226fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.979164 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2379a5365b7645f68ae12dc5feeb64eb', 'request_id': 'req-35782e1b-400d-47b0-9af6-4c2ab59226fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.995801 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] The token cache is being invalidate because project 2379a5365b7645f68ae12dc5feeb64eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:36.996093 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:36.996794 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.002239 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1234/2443] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:36 2022] DELETE /identity/v3/projects/2379a5365b7645f68ae12dc5feeb64eb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:37.002482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:37.003863 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.013009 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fecb9bd79a6e4f7480f0db0f6f6528ef'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.020621 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fecb9bd79a6e4f7480f0db0f6f6528ef'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.034184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fecb9bd79a6e4f7480f0db0f6f6528ef'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.034872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fecb9bd79a6e4f7480f0db0f6f6528ef'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.035492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-23d2f5ba-0881-4e1d-b37e-2a26a5ddcf83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fecb9bd79a6e4f7480f0db0f6f6528ef'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.054610 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1210/2444] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:36 2022] DELETE /identity/v3/users/fecb9bd79a6e4f7480f0db0f6f6528ef => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:37.059672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:37.059959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:37.060999 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:37.078523 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35782e1b-400d-47b0-9af6-4c2ab59226fc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:37.086448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:37.088939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.119975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:37.142551 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:37.143299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:37.143654 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:37.144077 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] PATH_INFO: `/v3/users/b14941613dd9429badcfe17095ba5cfe` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:37.145209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b14941613dd9429badcfe17095ba5cfe)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:37.147535 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:37.149264 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1211/2445] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:37.159570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:37.160963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.200333 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b14941613dd9429badcfe17095ba5cfe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.207272 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b14941613dd9429badcfe17095ba5cfe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.220612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b14941613dd9429badcfe17095ba5cfe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.222096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:37.227292 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee71b6-b0f4-45f9-b6e4-b18ae1d25375 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:37.230271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b14941613dd9429badcfe17095ba5cfe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.230962 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b14941613dd9429badcfe17095ba5cfe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.234004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:37.235696 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:37.236403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:37.237071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:37.237340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1235/2446] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:37 2022] DELETE /identity/v3/users/b14941613dd9429badcfe17095ba5cfe => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:37.240665 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.255471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:37.257442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:37.268020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1212/2447] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:47:37.695706 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:37.697333 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:37.698842 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.699141 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.711710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d9af-42ac-4d4e-acb0-59238ff468fc service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:37.779522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74eca68-d28b-44cd-aed7-d125563ec993 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:37.779807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:37.780453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:37.780742 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:37.781020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] PATH_INFO: `/v3/projects/811482ba1a6846b585f0a6b4fac3d9c3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:37.782176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811482ba1a6846b585f0a6b4fac3d9c3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:37.784156 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:37.784593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:37.785287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:37.785636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:37.786015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] PATH_INFO: `/v3/projects/479850a7717944929bfd825d3fc8a701` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:37.787208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=479850a7717944929bfd825d3fc8a701)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:37.789248 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:37.835958 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811482ba1a6846b585f0a6b4fac3d9c3', 'request_id': 'req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.842045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811482ba1a6846b585f0a6b4fac3d9c3', 'request_id': 'req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.848938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479850a7717944929bfd825d3fc8a701', 'request_id': 'req-770bf481-d885-466a-92b2-c33b9bcb3114'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.850339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811482ba1a6846b585f0a6b4fac3d9c3', 'request_id': 'req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.855714 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479850a7717944929bfd825d3fc8a701', 'request_id': 'req-770bf481-d885-466a-92b2-c33b9bcb3114'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.857534 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:37.862798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] The token cache is being invalidate because project 811482ba1a6846b585f0a6b4fac3d9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:37.862938 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479850a7717944929bfd825d3fc8a701', 'request_id': 'req-770bf481-d885-466a-92b2-c33b9bcb3114'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.863339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.863908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.865593 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1213/2448] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:37 2022] DELETE /identity/v3/projects/811482ba1a6846b585f0a6b4fac3d9c3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:37.870145 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:37.871727 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.881206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] The token cache is being invalidate because project 479850a7717944929bfd825d3fc8a701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:37.881500 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.882420 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-770bf481-d885-466a-92b2-c33b9bcb3114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:37.888113 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1236/2449] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:37 2022] DELETE /identity/v3/projects/479850a7717944929bfd825d3fc8a701 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:37.890742 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef1a41-ff14-4f9e-93ff-e842f7bcaf76 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:37.890945 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef1a41-ff14-4f9e-93ff-e842f7bcaf76 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:37.891576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef1a41-ff14-4f9e-93ff-e842f7bcaf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:37.916863 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c1ef1a41-ff14-4f9e-93ff-e842f7bcaf76 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:37.918267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c1ef1a41-ff14-4f9e-93ff-e842f7bcaf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:37.945277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5932c8-4cac-4eb5-ba7a-6d3a9702e684 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:37.974677 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:37.983297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1237/2450] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:37.990882 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ecd7-9976-4162-b861-4cebcaf5bd58 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:37.991242 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ecd7-9976-4162-b861-4cebcaf5bd58 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:37.991518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ecd7-9976-4162-b861-4cebcaf5bd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.009545 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.010271 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.010549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.010821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] PATH_INFO: `/v3/users/313f3ed12d6440418b0769d0a48146b5` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.011015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3ff9ecd7-9976-4162-b861-4cebcaf5bd58 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:38.012099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=313f3ed12d6440418b0769d0a48146b5)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.012297 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3ff9ecd7-9976-4162-b861-4cebcaf5bd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:38.014027 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.021120 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1238/2451] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:38.026260 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ecd7-9976-4162-b861-4cebcaf5bd58 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.035406 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ecd7-9976-4162-b861-4cebcaf5bd58 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.040136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.041520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.041860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.042142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.042778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:38.043091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.044675 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.059873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1239/2452] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:47:38.060841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.068701 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313f3ed12d6440418b0769d0a48146b5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.070765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1518a2-ba78-400a-b157-1e867ced9d98 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.075841 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.076552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.076672 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313f3ed12d6440418b0769d0a48146b5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.076830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.077102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.077737 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.079619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.089440 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313f3ed12d6440418b0769d0a48146b5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.090242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313f3ed12d6440418b0769d0a48146b5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.090970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313f3ed12d6440418b0769d0a48146b5'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.105506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1214/2453] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:37 2022] DELETE /identity/v3/users/313f3ed12d6440418b0769d0a48146b5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:38.112572 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1240/2454] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:38 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:38.117411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.119176 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:38.194304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a1129-efbf-48d3-975e-ddf90fdf5881 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.252343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:38.268813 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.270119 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.270541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.270835 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.272083 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.275443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.300926 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1215/2455] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:38 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:38.305385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.316674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3fe89-67b8-4d35-b867-2eb79a9b90f8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.321303 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.321965 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.322246 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.322521 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.323162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.324747 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.334765 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1241/2456] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:38.339039 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.348978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c6cd6-a838-4baf-ac23-077f916eda11 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.354115 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.354705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.354993 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.355276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.355919 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.357506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.367475 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1216/2457] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:38.372788 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.383025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e276d-27da-4a92-b793-77f710a0bd7e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.388104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a6c55-80d9-4dd6-b65a-21834c36ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.388941 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a6c55-80d9-4dd6-b65a-21834c36ba0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.389136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a6c55-80d9-4dd6-b65a-21834c36ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.389448 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a6c55-80d9-4dd6-b65a-21834c36ba0d admin admin] PATH_INFO: `/v3/projects/24521b04f6ec4b2ba29dca8730204b73/users/316dd70e52344ea08d4b09658f48f5e0/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.398535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.400987 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:38.408595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a6c55-80d9-4dd6-b65a-21834c36ba0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24521b04f6ec4b2ba29dca8730204b73, user_id=316dd70e52344ea08d4b09658f48f5e0, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.411574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a6c55-80d9-4dd6-b65a-21834c36ba0d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.420699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d25df-5a9d-477d-a5fb-09af0b95584c admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.426673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.427877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.428161 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.428444 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] PATH_INFO: `/v3/projects/c0b2081df01a4d4682c6fe028bcde02d` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.430082 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b2081df01a4d4682c6fe028bcde02d)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.432569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.433209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1242/2458] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:38 2022] PUT /identity/v3/projects/24521b04f6ec4b2ba29dca8730204b73/users/316dd70e52344ea08d4b09658f48f5e0/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:38.439163 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.439518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.439867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.461902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:38.463895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:38.493211 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b2081df01a4d4682c6fe028bcde02d', 'request_id': 'req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.510790 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b2081df01a4d4682c6fe028bcde02d', 'request_id': 'req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.539943 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b2081df01a4d4682c6fe028bcde02d', 'request_id': 'req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.576662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] The token cache is being invalidate because project c0b2081df01a4d4682c6fe028bcde02d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:38.576662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.576662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3e81cc7d-cf60-4b82-86fc-688f7d6ca743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:38.576662 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1217/2459] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:38 2022] DELETE /identity/v3/projects/c0b2081df01a4d4682c6fe028bcde02d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:38.578517 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc964-0540-433c-adeb-305ddfe65c81 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.578965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc964-0540-433c-adeb-305ddfe65c81 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.579308 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfc964-0540-433c-adeb-305ddfe65c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.595399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1243/2460] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:38.600266 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-94bfc964-0540-433c-adeb-305ddfe65c81 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:38.601481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-94bfc964-0540-433c-adeb-305ddfe65c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:38.605427 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.606868 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:38.621339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:38.666648 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1218/2461] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:38.668598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41a8a8-c33f-4e9c-88b0-561162d979c5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.674620 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee04df8-de93-4f4c-88ea-6e6c79b6c809 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.674827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee04df8-de93-4f4c-88ea-6e6c79b6c809 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.675104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee04df8-de93-4f4c-88ea-6e6c79b6c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.695220 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-3ee04df8-de93-4f4c-88ea-6e6c79b6c809 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:38.696603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-3ee04df8-de93-4f4c-88ea-6e6c79b6c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:38.705830 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1219/2462] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:38.709534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee04df8-de93-4f4c-88ea-6e6c79b6c809 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.720143 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee04df8-de93-4f4c-88ea-6e6c79b6c809 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.723696 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.724304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.724596 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.724858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.726978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.727603 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.728598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.728936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.729652 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:38.730003 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.731592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.735659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.737081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.747095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1244/2463] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:47:38.749681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1220/2464] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:47:38.753862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.764208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ecca6-0cae-4f80-954a-e3f39360f50e service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.770510 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.771612 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.772060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.772752 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.773700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.777784 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.812423 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1245/2465] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:38 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:38.817269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.827520 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089254f-7eb2-4d33-9d04-8abbc42d033a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.865096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:38.891991 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.892604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.892893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.893170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.893829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.895745 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.919142 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1221/2466] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:38 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:38.924105 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.937421 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333aa06-317a-496f-814d-17f058991b1c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.942261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.942871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.943157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.943441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.944353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.946270 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.957617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1246/2467] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:38.961676 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:38.972797 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63943d8-6199-4716-a9aa-19bc03d15daa admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:38.977527 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f511785-c99a-403d-98af-276f0a1f01b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:38.979034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511785-c99a-403d-98af-276f0a1f01b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:38.979313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511785-c99a-403d-98af-276f0a1f01b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:38.979589 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f511785-c99a-403d-98af-276f0a1f01b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:38.980230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f511785-c99a-403d-98af-276f0a1f01b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:38.982262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f511785-c99a-403d-98af-276f0a1f01b5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:38.992786 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1222/2468] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:38.997000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.008112 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982edbf0-0a59-4cc4-84e9-8c9aad9e0673 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.012880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.013569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.013873 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.014168 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] PATH_INFO: `/v3/projects/a476879b92454d80b42d286a24803548/users/6a1d5201d746406d90778ea24e3ab8d7/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.029504 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a476879b92454d80b42d286a24803548, user_id=6a1d5201d746406d90778ea24e3ab8d7, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.032231 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.049415 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1247/2469] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:38 2022] PUT /identity/v3/projects/a476879b92454d80b42d286a24803548/users/6a1d5201d746406d90778ea24e3ab8d7/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:39.062133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c83bd7-d1d6-4f3b-9571-9d8c912612ad None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.062592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c83bd7-d1d6-4f3b-9571-9d8c912612ad None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.062975 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c83bd7-d1d6-4f3b-9571-9d8c912612ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.095812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-35c83bd7-d1d6-4f3b-9571-9d8c912612ad None None] MFA Rules not processed for user `6a1d5201d746406d90778ea24e3ab8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:39.097117 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-35c83bd7-d1d6-4f3b-9571-9d8c912612ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:39.176972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1223/2470] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:39.181858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.247545 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b4d03-977f-476d-b7d1-ebe1d227f494 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.252780 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.253385 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.253656 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.253956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.254593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.256468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.278461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:39.286472 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1248/2471] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:39 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:39.291934 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c83bd7-d1d6-4f3b-9571-9d8c912612ad None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.303235 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c83bd7-d1d6-4f3b-9571-9d8c912612ad None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.374549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e88c9-0af6-477d-8577-3ae19d0eedef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.375170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e88c9-0af6-477d-8577-3ae19d0eedef admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.375454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e88c9-0af6-477d-8577-3ae19d0eedef admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.375728 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e88c9-0af6-477d-8577-3ae19d0eedef admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.376377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e88c9-0af6-477d-8577-3ae19d0eedef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.378335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e88c9-0af6-477d-8577-3ae19d0eedef admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.404175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1224/2472] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:39 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:39.408136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.421026 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d63981-b6c7-4c1e-8893-f6e7ad9ee95d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.432637 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.432887 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.433198 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.433505 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.434188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.434666 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.435091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.435901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.437648 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.450271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1249/2473] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:39.454352 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.467867 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a785966-9c58-41a8-9138-0eb740f6faa8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.468336 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] MFA Rules not processed for user `316dd70e52344ea08d4b09658f48f5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:39.469552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:39.473680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.474490 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.474790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.475060 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] PATH_INFO: `/v3/projects/611e4cb290a44d34ad1072a32a981173/users/746d7c3fa1314212866f530c8394d35f/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.492732 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611e4cb290a44d34ad1072a32a981173, user_id=746d7c3fa1314212866f530c8394d35f, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.495716 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.498354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:39.511988 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1250/2474] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:39 2022] PUT /identity/v3/projects/611e4cb290a44d34ad1072a32a981173/users/746d7c3fa1314212866f530c8394d35f/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:39.516027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.553569 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1225/2475] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:39.559004 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.560700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:39.574204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d1604-ce38-431d-8237-dbc6ec507917 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.620209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d496e-8216-4a18-98c6-8f0e783648f5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.632364 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.633198 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.633481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.637061 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.637977 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.640316 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.662088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.662088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.662088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.662088 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.662384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.662384 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.669983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1251/2476] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:39.674206 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.685244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6c9d4-8aff-4bb8-a6e5-b533b3186819 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.690092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840867a-9c62-4b42-a85c-2431fb5a5c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.690999 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840867a-9c62-4b42-a85c-2431fb5a5c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.691290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840867a-9c62-4b42-a85c-2431fb5a5c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.691393 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1226/2477] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:39 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:39.691613 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840867a-9c62-4b42-a85c-2431fb5a5c61 admin admin] PATH_INFO: `/v3/projects/611e4cb290a44d34ad1072a32a981173/users/746d7c3fa1314212866f530c8394d35f/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.693645 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840867a-9c62-4b42-a85c-2431fb5a5c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611e4cb290a44d34ad1072a32a981173, user_id=746d7c3fa1314212866f530c8394d35f, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.695960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.696357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840867a-9c62-4b42-a85c-2431fb5a5c61 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.707619 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1252/2478] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:39 2022] PUT /identity/v3/projects/611e4cb290a44d34ad1072a32a981173/users/746d7c3fa1314212866f530c8394d35f/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:39.718659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.718948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.719228 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.747092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] MFA Rules not processed for user `746d7c3fa1314212866f530c8394d35f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:39.748296 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:39.763597 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac148c-f8a6-4292-a892-2992c0a655a4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.819456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:39.826913 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.827568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.828054 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.828438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.829330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.831495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.834697 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1253/2479] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:39.846274 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.848213 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:39.860173 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1227/2480] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:39 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:39.864289 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.874240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782cc8-d962-4190-9475-ddc70cb51775 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.879313 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.879921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.880596 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.880880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.881586 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.883272 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.888857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:39.898134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1228/2481] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:39.899396 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:39.911296 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306ded9-76b2-4c02-974e-3fbea9810f79 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.916482 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.917277 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.917639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.918035 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] PATH_INFO: `/v3/projects/51a6bb938d7c431893f3375fe6495997/users/3329bbe77b3d4a878a0d2d26787429e2/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.926991 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed60e4b-b0e2-4495-afc4-a530ca353a8d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:39.930985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:39.931641 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:39.931917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:39.932188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:39.934246 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a6bb938d7c431893f3375fe6495997, user_id=3329bbe77b3d4a878a0d2d26787429e2, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.936957 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.942269 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:39.943840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:39.950851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1229/2482] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:39 2022] PUT /identity/v3/projects/51a6bb938d7c431893f3375fe6495997/users/3329bbe77b3d4a878a0d2d26787429e2/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:39.954652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1254/2483] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:39.954851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:40.020034 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b38c9-0530-4ba7-a667-f52d8dc51772 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:40.026052 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:40.026659 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:40.026937 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:40.027209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:40.027849 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:40.029423 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:40.041801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1230/2484] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:40.045755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:40.056560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c54f3e-b8f5-42d0-902e-278e281ae45f service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:40.061229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07adad-9970-47a6-b396-1d02c3e00059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:40.061937 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07adad-9970-47a6-b396-1d02c3e00059 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:40.062210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07adad-9970-47a6-b396-1d02c3e00059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:40.062481 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07adad-9970-47a6-b396-1d02c3e00059 admin admin] PATH_INFO: `/v3/projects/51a6bb938d7c431893f3375fe6495997/users/3329bbe77b3d4a878a0d2d26787429e2/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:40.064635 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07adad-9970-47a6-b396-1d02c3e00059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a6bb938d7c431893f3375fe6495997, user_id=3329bbe77b3d4a878a0d2d26787429e2, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:40.064871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:40.067275 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07adad-9970-47a6-b396-1d02c3e00059 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:40.079267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1255/2485] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:40 2022] PUT /identity/v3/projects/51a6bb938d7c431893f3375fe6495997/users/3329bbe77b3d4a878a0d2d26787429e2/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:40.123509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a3eaa-68ae-4460-819a-5449c52c66f2 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:40.176980 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:40.177700 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:40.178138 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:40.178417 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:40.202448 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:40.256131 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:40.257935 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:40.270412 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1231/2486] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:47:40.790778 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72745e38-7ebf-4969-a95d-c225984ce809 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:40.791286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72745e38-7ebf-4969-a95d-c225984ce809 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:40.791526 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72745e38-7ebf-4969-a95d-c225984ce809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:40.820789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-72745e38-7ebf-4969-a95d-c225984ce809 None None] MFA Rules not processed for user `3329bbe77b3d4a878a0d2d26787429e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:40.822966 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-72745e38-7ebf-4969-a95d-c225984ce809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:40.905114 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1256/2487] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:40.917994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:40.919590 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:40.935053 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d6327-2122-4cef-8f11-ff65f87f3647 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:40.938594 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:40.939224 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:40.939504 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:40.939780 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:41.000626 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:41.002184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:41.011893 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1232/2488] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:42.395015 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72745e38-7ebf-4969-a95d-c225984ce809 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:42.405368 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72745e38-7ebf-4969-a95d-c225984ce809 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:42.408950 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac481b-c814-479b-ab59-94df507860e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:42.409547 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac481b-c814-479b-ab59-94df507860e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:42.409862 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac481b-c814-479b-ab59-94df507860e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:42.410137 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac481b-c814-479b-ab59-94df507860e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:42.419541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac481b-c814-479b-ab59-94df507860e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:42.420970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac481b-c814-479b-ab59-94df507860e5 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:42.430287 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1257/2489] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:42.915860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:42.984427 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191a49f-d823-498e-a33c-fc71f00f7484 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:42.989984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:42.990761 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:42.991045 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:42.991474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] PATH_INFO: `/v3/users/6a1d5201d746406d90778ea24e3ab8d7` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:42.992616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a1d5201d746406d90778ea24e3ab8d7)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:42.995585 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:43.053299 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a1d5201d746406d90778ea24e3ab8d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.060535 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a1d5201d746406d90778ea24e3ab8d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.062434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:43.074170 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a1d5201d746406d90778ea24e3ab8d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.074859 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a1d5201d746406d90778ea24e3ab8d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.075578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a1d5201d746406d90778ea24e3ab8d7'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.088719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1233/2490] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:42 2022] DELETE /identity/v3/users/6a1d5201d746406d90778ea24e3ab8d7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:43.093267 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:43.093553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:43.093872 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:43.112618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:43.113893 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:43.171883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1258/2491] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:43.182200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:43.183588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:43.241199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68adb5-26ab-41a6-81fe-7eb957af837f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:43.245007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:43.245604 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:43.245923 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:43.246204 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:43.258731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:43.261075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:43.273197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1234/2492] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:43.542136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:43.543685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:43.559404 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238410b0-e9ad-4e84-b401-1aa15267ddf8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:43.564975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:43.565620 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:43.565930 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:43.566203 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] PATH_INFO: `/v3/projects/a476879b92454d80b42d286a24803548` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:43.567339 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a476879b92454d80b42d286a24803548)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:43.569309 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:43.608679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:43.622712 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a476879b92454d80b42d286a24803548', 'request_id': 'req-4d0d575b-9869-449b-aabf-87ab1012039e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.628059 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a476879b92454d80b42d286a24803548', 'request_id': 'req-4d0d575b-9869-449b-aabf-87ab1012039e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.634653 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a476879b92454d80b42d286a24803548', 'request_id': 'req-4d0d575b-9869-449b-aabf-87ab1012039e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.647214 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] The token cache is being invalidate because project a476879b92454d80b42d286a24803548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:43.647498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.648104 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.650294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1259/2493] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:43 2022] DELETE /identity/v3/projects/a476879b92454d80b42d286a24803548 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:43.654321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:43.663760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be735736-ec68-453a-af19-5ddd88ed6892 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:43.735726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:43.735726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:43.735726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:43.735726 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] PATH_INFO: `/v3/users/746d7c3fa1314212866f530c8394d35f` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:43.736242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746d7c3fa1314212866f530c8394d35f)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:43.738438 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:43.795821 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746d7c3fa1314212866f530c8394d35f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.803190 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746d7c3fa1314212866f530c8394d35f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.818673 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746d7c3fa1314212866f530c8394d35f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.820151 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746d7c3fa1314212866f530c8394d35f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.821010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3725d497-a48d-4672-b92a-844df23feab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746d7c3fa1314212866f530c8394d35f'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:43.834808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:43.845070 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1235/2494] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:43 2022] DELETE /identity/v3/users/746d7c3fa1314212866f530c8394d35f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:44.166621 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.168149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:44.184056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d575b-9869-449b-aabf-87ab1012039e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.251804 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.252539 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.252887 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.253229 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] PATH_INFO: `/v3/projects/611e4cb290a44d34ad1072a32a981173` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.254498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=611e4cb290a44d34ad1072a32a981173)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.256492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.306473 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611e4cb290a44d34ad1072a32a981173', 'request_id': 'req-e0aa0705-3e32-41cc-9764-0b550495df38'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:44.312356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611e4cb290a44d34ad1072a32a981173', 'request_id': 'req-e0aa0705-3e32-41cc-9764-0b550495df38'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:44.320050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611e4cb290a44d34ad1072a32a981173', 'request_id': 'req-e0aa0705-3e32-41cc-9764-0b550495df38'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:44.331810 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] The token cache is being invalidate because project 611e4cb290a44d34ad1072a32a981173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:44.332098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:44.332720 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0aa0705-3e32-41cc-9764-0b550495df38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:44.334247 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1260/2495] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:44 2022] DELETE /identity/v3/projects/611e4cb290a44d34ad1072a32a981173 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:44.339311 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5b1f-d77a-4c38-b06b-da63c406647d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.339808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5b1f-d77a-4c38-b06b-da63c406647d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.340005 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5b1f-d77a-4c38-b06b-da63c406647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.361026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6aee5b1f-d77a-4c38-b06b-da63c406647d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:44.362342 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6aee5b1f-d77a-4c38-b06b-da63c406647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:44.424522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1236/2496] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:44.433006 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864960e2-2d3b-4208-8ac0-6c7be9ffcd46 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.433290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864960e2-2d3b-4208-8ac0-6c7be9ffcd46 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.433560 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864960e2-2d3b-4208-8ac0-6c7be9ffcd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.454133 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-864960e2-2d3b-4208-8ac0-6c7be9ffcd46 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:44.455461 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-864960e2-2d3b-4208-8ac0-6c7be9ffcd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:44.463546 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1261/2497] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:44.467869 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5b1f-d77a-4c38-b06b-da63c406647d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.478128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5b1f-d77a-4c38-b06b-da63c406647d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.482775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.483370 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.483650 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.483924 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.484570 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:44.484875 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.486521 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.498499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1237/2498] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:47:44.502590 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864960e2-2d3b-4208-8ac0-6c7be9ffcd46 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.511833 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864960e2-2d3b-4208-8ac0-6c7be9ffcd46 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.516787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.517394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.517722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.517972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.518604 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.520626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.553834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1262/2499] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:44 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:44.558661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.568516 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e344c0-7cc4-4197-b181-8bf08c128568 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.624104 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.624731 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.625010 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.625281 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.625950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.627847 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.648576 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1238/2500] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:44 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:44.653188 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.663495 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1067de5-3866-44ff-8956-329f4f1ecf28 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.669035 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.669627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.669936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.670237 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.670880 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.672458 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.682964 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1263/2501] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:44.686734 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.696457 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5619023-e462-40e1-8721-5a931f7d73ae admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.701331 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bf7de-1326-4838-9990-807b9ed5f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.701962 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf7de-1326-4838-9990-807b9ed5f06f admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.702360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf7de-1326-4838-9990-807b9ed5f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.702699 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf7de-1326-4838-9990-807b9ed5f06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.703401 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bf7de-1326-4838-9990-807b9ed5f06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.705044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bf7de-1326-4838-9990-807b9ed5f06f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.716334 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1239/2502] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:44.720529 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.731271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb8969-bb73-479f-8763-7ce01e79a32c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.735928 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.736570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.736852 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.737127 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] PATH_INFO: `/v3/projects/166070ca09554fd48b9d4447324d0c3b/users/a6094f5b13d0490c9e596eefbbfdaaf2/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.740195 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:44.752902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=166070ca09554fd48b9d4447324d0c3b, user_id=a6094f5b13d0490c9e596eefbbfdaaf2, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.755573 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.767777 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1264/2503] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:44 2022] PUT /identity/v3/projects/166070ca09554fd48b9d4447324d0c3b/users/a6094f5b13d0490c9e596eefbbfdaaf2/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:44.779984 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8938cc-a356-47c6-83b1-d7cb2b9beca3 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.780276 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8938cc-a356-47c6-83b1-d7cb2b9beca3 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.780557 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8938cc-a356-47c6-83b1-d7cb2b9beca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.809431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-1c8938cc-a356-47c6-83b1-d7cb2b9beca3 None None] MFA Rules not processed for user `a6094f5b13d0490c9e596eefbbfdaaf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:44.810721 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-1c8938cc-a356-47c6-83b1-d7cb2b9beca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:44.871291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:44.882548 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1240/2504] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:44.893543 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:44.895225 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:44.954108 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3556bd6-e1b0-4822-8321-938c4b659942 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:44.957684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33453264-5b5b-4d3a-a525-3dfc09b3d556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:44.958315 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33453264-5b5b-4d3a-a525-3dfc09b3d556 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:44.958591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33453264-5b5b-4d3a-a525-3dfc09b3d556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:44.958860 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33453264-5b5b-4d3a-a525-3dfc09b3d556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:44.967835 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33453264-5b5b-4d3a-a525-3dfc09b3d556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:44.969251 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33453264-5b5b-4d3a-a525-3dfc09b3d556 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:44.978639 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1265/2505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:53.456229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8938cc-a356-47c6-83b1-d7cb2b9beca3 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:53.521244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8938cc-a356-47c6-83b1-d7cb2b9beca3 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:53.526812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:53.527155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:53.527442 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:53.527720 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] PATH_INFO: `/v3/users/a6094f5b13d0490c9e596eefbbfdaaf2` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:53.529099 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6094f5b13d0490c9e596eefbbfdaaf2)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:53.531450 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:53.593565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6094f5b13d0490c9e596eefbbfdaaf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:53.601549 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6094f5b13d0490c9e596eefbbfdaaf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:53.615575 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6094f5b13d0490c9e596eefbbfdaaf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:53.616452 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6094f5b13d0490c9e596eefbbfdaaf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:53.617061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6094f5b13d0490c9e596eefbbfdaaf2'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:53.630702 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1241/2506] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:53 2022] DELETE /identity/v3/users/a6094f5b13d0490c9e596eefbbfdaaf2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:53.635722 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:53.636234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:53.636417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:53.656151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:53.657350 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:53.716598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1266/2507] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:53.726764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:53.728437 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:53.798845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5b27c-99f0-475d-bff9-9e398aaea89e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:53.798845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272572d6-70ac-4e37-a184-09b0bcb49200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:53.798845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272572d6-70ac-4e37-a184-09b0bcb49200 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:53.798845 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272572d6-70ac-4e37-a184-09b0bcb49200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:53.799214 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272572d6-70ac-4e37-a184-09b0bcb49200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:53.807705 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272572d6-70ac-4e37-a184-09b0bcb49200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:53.809128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272572d6-70ac-4e37-a184-09b0bcb49200 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:53.818592 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1242/2508] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:54.090467 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.092107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:54.104851 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8f06-057f-4630-a03c-5034da41a357 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.110156 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.110923 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.111559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.111559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] PATH_INFO: `/v3/projects/166070ca09554fd48b9d4447324d0c3b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.112840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=166070ca09554fd48b9d4447324d0c3b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.114951 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.162306 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166070ca09554fd48b9d4447324d0c3b', 'request_id': 'req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:54.168541 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166070ca09554fd48b9d4447324d0c3b', 'request_id': 'req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:54.174661 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166070ca09554fd48b9d4447324d0c3b', 'request_id': 'req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:54.187708 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] The token cache is being invalidate because project 166070ca09554fd48b9d4447324d0c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:54.187993 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:54.188601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f42f3e9-1f84-4b04-b8a0-34e8f3141f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:54.190096 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1267/2509] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:54 2022] DELETE /identity/v3/projects/166070ca09554fd48b9d4447324d0c3b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:54.195085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033a011-d6da-4275-b7e1-290e3bcb1e65 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.195375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033a011-d6da-4275-b7e1-290e3bcb1e65 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.195654 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033a011-d6da-4275-b7e1-290e3bcb1e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.214904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-b033a011-d6da-4275-b7e1-290e3bcb1e65 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:54.216096 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-b033a011-d6da-4275-b7e1-290e3bcb1e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:54.225323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:54.278454 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1243/2510] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:47:54.286216 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa675b-5307-492f-a817-9f2c21d94c6b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.286501 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa675b-5307-492f-a817-9f2c21d94c6b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.286779 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa675b-5307-492f-a817-9f2c21d94c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.307528 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b7aa675b-5307-492f-a817-9f2c21d94c6b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:54.308700 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b7aa675b-5307-492f-a817-9f2c21d94c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:54.316430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1268/2511] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:54.320607 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033a011-d6da-4275-b7e1-290e3bcb1e65 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.330248 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033a011-d6da-4275-b7e1-290e3bcb1e65 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.335871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.336459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.337155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.337155 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.337900 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:54.338210 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.339823 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.351114 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1244/2512] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:47:54.355920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa675b-5307-492f-a817-9f2c21d94c6b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.361419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:54.365201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa675b-5307-492f-a817-9f2c21d94c6b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.371827 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.372592 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.372989 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.373341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.374314 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.376187 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.404279 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1269/2513] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:54 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:54.408643 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.419579 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ad67a-3d66-4fdc-98e6-3ef832c77a3e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.479357 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.479950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.480254 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.480509 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.481146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.483076 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.504783 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1245/2514] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:54 2022] POST /identity/v3/users => generated 281 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:54.508939 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.519970 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ada21-591c-4840-894c-dc0327c08b23 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.529004 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.529601 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.529915 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.530194 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.530891 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.532494 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.544046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1270/2515] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:54.547966 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.557963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f738f-0382-4999-8abb-8768dcddbcdd admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.562538 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77b6aa-8682-4acd-ad3f-83195aea43d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.563128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b6aa-8682-4acd-ad3f-83195aea43d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.563404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b6aa-8682-4acd-ad3f-83195aea43d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.563675 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b6aa-8682-4acd-ad3f-83195aea43d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.564309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77b6aa-8682-4acd-ad3f-83195aea43d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.565907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77b6aa-8682-4acd-ad3f-83195aea43d9 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.576234 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1246/2516] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:54.580426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.590822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e2fe0-3801-42b8-a4d1-e521bcf345f0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.595379 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.596018 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.596292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.596562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] PATH_INFO: `/v3/projects/34359b87c17e47098dd898524c707c49/users/a633d42f064d4f72a86ab18cbe4c5576/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.611578 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34359b87c17e47098dd898524c707c49, user_id=a633d42f064d4f72a86ab18cbe4c5576, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.614311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.627960 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1271/2517] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:54 2022] PUT /identity/v3/projects/34359b87c17e47098dd898524c707c49/users/a633d42f064d4f72a86ab18cbe4c5576/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:54.638372 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f053-323b-4012-b93d-c76ba61bbbc2 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.638657 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f053-323b-4012-b93d-c76ba61bbbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.638929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f053-323b-4012-b93d-c76ba61bbbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.668217 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-5366f053-323b-4012-b93d-c76ba61bbbc2 None None] MFA Rules not processed for user `a633d42f064d4f72a86ab18cbe4c5576`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:54.669459 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5366f053-323b-4012-b93d-c76ba61bbbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:54.731968 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1247/2518] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:54.766574 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.768440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:54.828848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e6493-9e3e-4cf1-a64f-58fcfa952c63 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.832568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9abc-293c-4029-a73e-dc940e635cb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.833162 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9abc-293c-4029-a73e-dc940e635cb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.833438 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9abc-293c-4029-a73e-dc940e635cb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.833707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9abc-293c-4029-a73e-dc940e635cb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.843159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c9abc-293c-4029-a73e-dc940e635cb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.844585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c9abc-293c-4029-a73e-dc940e635cb8 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:54.853667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1272/2519] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:54.870354 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366f053-323b-4012-b93d-c76ba61bbbc2 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:54.883362 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:54.936974 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366f053-323b-4012-b93d-c76ba61bbbc2 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:54.942656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:54.943428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:54.943753 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:54.944075 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] PATH_INFO: `/v3/users/a633d42f064d4f72a86ab18cbe4c5576` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:54.945326 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a633d42f064d4f72a86ab18cbe4c5576)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:54.948127 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:55.002787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a633d42f064d4f72a86ab18cbe4c5576'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.010730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a633d42f064d4f72a86ab18cbe4c5576'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.024216 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a633d42f064d4f72a86ab18cbe4c5576'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.025518 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a633d42f064d4f72a86ab18cbe4c5576'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.026553 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a633d42f064d4f72a86ab18cbe4c5576'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.043902 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1248/2520] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:54 2022] DELETE /identity/v3/users/a633d42f064d4f72a86ab18cbe4c5576 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:55.048489 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:55.049102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:55.049455 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:55.074920 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:55.076221 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:55.146597 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1273/2521] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:55.157410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:55.158851 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:55.216322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa242c-5a24-4996-84b6-4ec9d69636ca admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:55.220061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e96d5-ad98-4652-89ea-b7c3cbbcaa9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:55.220697 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e96d5-ad98-4652-89ea-b7c3cbbcaa9d service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:55.220972 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e96d5-ad98-4652-89ea-b7c3cbbcaa9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:55.221242 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e96d5-ad98-4652-89ea-b7c3cbbcaa9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:55.230587 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e96d5-ad98-4652-89ea-b7c3cbbcaa9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:55.232012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e96d5-ad98-4652-89ea-b7c3cbbcaa9d service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:55.241758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1249/2522] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:55.754145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:55.759383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:55.776441 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618263f-c21a-4023-9be7-50d23c651833 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:55.782569 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:55.783201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:55.783497 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:55.783759 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] PATH_INFO: `/v3/projects/34359b87c17e47098dd898524c707c49` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:55.784946 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34359b87c17e47098dd898524c707c49)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:55.786783 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:55.838673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:55.840065 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34359b87c17e47098dd898524c707c49', 'request_id': 'req-16e88a06-2ced-4080-9078-301bce89d207'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.847089 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34359b87c17e47098dd898524c707c49', 'request_id': 'req-16e88a06-2ced-4080-9078-301bce89d207'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.855871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34359b87c17e47098dd898524c707c49', 'request_id': 'req-16e88a06-2ced-4080-9078-301bce89d207'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.871259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] The token cache is being invalidate because project 34359b87c17e47098dd898524c707c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:47:55.871416 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.872091 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-16e88a06-2ced-4080-9078-301bce89d207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:55.873659 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1274/2523] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:47:55 2022] DELETE /identity/v3/projects/34359b87c17e47098dd898524c707c49 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:55.878480 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af81c1-2ab8-4d5d-9fe2-c91e128e2a0b None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:55.878946 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af81c1-2ab8-4d5d-9fe2-c91e128e2a0b None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:55.879058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af81c1-2ab8-4d5d-9fe2-c91e128e2a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:55.900639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-84af81c1-2ab8-4d5d-9fe2-c91e128e2a0b None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:55.901960 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-84af81c1-2ab8-4d5d-9fe2-c91e128e2a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:55.932963 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:55.965862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1250/2524] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:55.973376 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ebe91-d43d-407e-8a6b-b981f2db3769 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:55.973667 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ebe91-d43d-407e-8a6b-b981f2db3769 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:55.973987 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ebe91-d43d-407e-8a6b-b981f2db3769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:55.994605 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-424ebe91-d43d-407e-8a6b-b981f2db3769 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:55.995805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-424ebe91-d43d-407e-8a6b-b981f2db3769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:56.003822 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1275/2525] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:56.007978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af81c1-2ab8-4d5d-9fe2-c91e128e2a0b None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.017891 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af81c1-2ab8-4d5d-9fe2-c91e128e2a0b None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.022857 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.023561 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.023843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.024644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.025356 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:47:56.025595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.027260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.041322 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1251/2526] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:47:56.044411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ebe91-d43d-407e-8a6b-b981f2db3769 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.053790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ebe91-d43d-407e-8a6b-b981f2db3769 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.059081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.059798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.060083 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.060349 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.060985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.062883 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.091244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1276/2527] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:56 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:56.095199 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.105490 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f1d5-d88d-4a5b-8f04-d33f991fee74 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.182502 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.183144 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.183431 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.183712 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.184369 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.186571 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.217339 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1252/2528] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:56 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:56.218826 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.230068 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159f87c-d20c-45fb-a864-cd3483c59c87 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.235000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.235626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.235901 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.236175 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.236824 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.238518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.250293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1277/2529] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:56.254775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.268388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38db62-74aa-4fe8-9b45-7095ed967194 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.273959 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaae64-e164-413a-bca8-e694a68d8399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.274741 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaae64-e164-413a-bca8-e694a68d8399 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.275152 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaae64-e164-413a-bca8-e694a68d8399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.275519 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaae64-e164-413a-bca8-e694a68d8399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.276414 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eaae64-e164-413a-bca8-e694a68d8399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.278122 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eaae64-e164-413a-bca8-e694a68d8399 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.292588 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1253/2530] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:56.296729 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.313914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1cd-446d-411f-bff4-7f34c05a6dc7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.313914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.313914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.313914 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.314400 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] PATH_INFO: `/v3/projects/dd18cba2516042da8a352a3e85faa548/users/9c550b766a3d4ba8839e2f9e8308beb3/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.331143 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd18cba2516042da8a352a3e85faa548, user_id=9c550b766a3d4ba8839e2f9e8308beb3, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.333800 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.351487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1278/2531] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:56 2022] PUT /identity/v3/projects/dd18cba2516042da8a352a3e85faa548/users/9c550b766a3d4ba8839e2f9e8308beb3/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:56.362283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.362572 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.362846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.401687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] MFA Rules not processed for user `9c550b766a3d4ba8839e2f9e8308beb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:56.401687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:56.474976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1254/2532] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:56.476149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.555116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6431ff-8af1-4704-bfbd-7d3dc3bb5016 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.560688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.561518 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.562479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.562479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.564157 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.573110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.617491 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.619253 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:56.621674 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1279/2533] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:56 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:56.625402 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.685832 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:56.708348 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:56.714017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c010b-0c48-464b-89fd-64d9d5b5e9bc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.720189 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.721079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.721552 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.722318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.723099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.725282 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.753768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1280/2534] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:56 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:56.756947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.758979 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f5a04-ec99-4fa1-8228-3095da2ffd36 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.767482 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa477053-4bfd-4c2c-88be-ac22d91b4882 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.772858 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.773449 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.773942 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.775031 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.775746 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.777515 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.789262 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1281/2535] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:47:56.793503 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.803753 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeed0ed-9d6b-4f0c-837e-cb56f3aaff1b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.809820 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.810488 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.810770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.811050 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.811805 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.813383 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.824185 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1282/2536] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:56.825827 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9af13-0eeb-4f13-9818-80c8054e062f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.826456 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9af13-0eeb-4f13-9818-80c8054e062f service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.826722 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9af13-0eeb-4f13-9818-80c8054e062f service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.827012 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9af13-0eeb-4f13-9818-80c8054e062f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.828413 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:56.829113 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-96f9af13-0eeb-4f13-9818-80c8054e062f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:56.839534 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a6cab-7160-4629-b7fe-fbccb93e13d9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:56.844335 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9af13-0eeb-4f13-9818-80c8054e062f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.844559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:56.845331 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.845668 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.845979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] PATH_INFO: `/v3/projects/2204bb7b6f5149568110babf8342258f/users/cd365434783a46bd90e6da9194b65ed6/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.846894 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9af13-0eeb-4f13-9818-80c8054e062f service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.857683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1255/2537] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:47:56.864365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2204bb7b6f5149568110babf8342258f, user_id=cd365434783a46bd90e6da9194b65ed6, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:56.864892 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:56.882477 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1283/2538] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:56 2022] PUT /identity/v3/projects/2204bb7b6f5149568110babf8342258f/users/cd365434783a46bd90e6da9194b65ed6/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:56.894015 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7906-39fb-4938-84cd-0bd40f434afb None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:56.894483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7906-39fb-4938-84cd-0bd40f434afb None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:56.895843 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c7906-39fb-4938-84cd-0bd40f434afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:56.928498 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-f02c7906-39fb-4938-84cd-0bd40f434afb None None] MFA Rules not processed for user `cd365434783a46bd90e6da9194b65ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:56.929683 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-f02c7906-39fb-4938-84cd-0bd40f434afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:57.008445 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1256/2539] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:57.013000 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:57.083502 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cade425-42c8-4035-8172-51b6456ddfbe admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:57.089589 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:57.090243 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.090520 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.090790 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.091505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:57.093346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:57.120585 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1284/2540] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:47:57 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:47:57.124954 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c7906-39fb-4938-84cd-0bd40f434afb None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:57.138165 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c7906-39fb-4938-84cd-0bd40f434afb None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:57.191105 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=100 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:57.202300 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:57.203126 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.203404 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.203678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.204321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:57.206398 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:57.230950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1257/2541] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:47:57 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:47:57.234551 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:57.247106 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d1a63-4310-40be-9438-63214210b706 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:57.251919 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:57.252564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.252874 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.253160 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.253917 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:57.255626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:57.268300 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1285/2542] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:57.272710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:57.283863 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6bd0c3-b006-45ee-8ce9-72f8e3e3fd7a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:57.290256 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:57.291229 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.291506 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.291775 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] PATH_INFO: `/v3/projects/7619df6754db47288ee1c376d5fe2202/users/7a3cfd5dccc24eb48a0640102ef2b60d/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.308066 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7619df6754db47288ee1c376d5fe2202, user_id=7a3cfd5dccc24eb48a0640102ef2b60d, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:57.310764 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:57.324560 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1258/2543] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:57 2022] PUT /identity/v3/projects/7619df6754db47288ee1c376d5fe2202/users/7a3cfd5dccc24eb48a0640102ef2b60d/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:47:57.328564 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:57.339889 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72344aeb-9de0-4282-9ecf-bc4d6ed95d22 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:57.400998 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77a9db-76dd-4dc6-973f-cb4d84e047b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:57.401627 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77a9db-76dd-4dc6-973f-cb4d84e047b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.401931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77a9db-76dd-4dc6-973f-cb4d84e047b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.402226 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77a9db-76dd-4dc6-973f-cb4d84e047b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.402832 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77a9db-76dd-4dc6-973f-cb4d84e047b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:57.404434 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77a9db-76dd-4dc6-973f-cb4d84e047b3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:57.415311 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1286/2544] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:47:57.419419 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:57.430020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f5b4c-56cc-4e7a-b810-a6eeab680585 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:57.434758 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:57.435422 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.435681 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.435950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] PATH_INFO: `/v3/projects/7619df6754db47288ee1c376d5fe2202/users/7a3cfd5dccc24eb48a0640102ef2b60d/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.437950 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7619df6754db47288ee1c376d5fe2202, user_id=7a3cfd5dccc24eb48a0640102ef2b60d, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:57.440862 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:57.452285 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1259/2545] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:47:57 2022] PUT /identity/v3/projects/7619df6754db47288ee1c376d5fe2202/users/7a3cfd5dccc24eb48a0640102ef2b60d/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:47:57.462631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.462902 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.463173 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.482342 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:57.493426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] MFA Rules not processed for user `7a3cfd5dccc24eb48a0640102ef2b60d`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:47:57.494791 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:57.568576 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1287/2546] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:47:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:47:57.580330 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:57.582051 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:57.641608 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a753a-072f-4fae-bca6-2ee407e01851 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:57.645275 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:57.645908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:57.646186 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:57.646713 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:57.703207 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:57.704636 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:57.714524 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1260/2547] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:59.277505 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:59.279149 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:59.293181 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700973c-45b8-4a19-a0d0-cbacff6cf356 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:59.297199 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:59.297979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:59.298340 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:59.298715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:59.377084 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:59.378857 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:59.393935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1288/2548] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:47:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:47:59.635818 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:47:59.638856 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:47:59.732058 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:47:59.737551 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658c6e8-7e68-4a7e-acb2-3c39d00b691e service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:47:59.742815 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:47:59.743801 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:47:59.744098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:47:59.744828 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] PATH_INFO: `/v3/users/316dd70e52344ea08d4b09658f48f5e0` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:47:59.745661 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=316dd70e52344ea08d4b09658f48f5e0)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:47:59.747877 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:47:59.831108 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316dd70e52344ea08d4b09658f48f5e0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:59.838971 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316dd70e52344ea08d4b09658f48f5e0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:59.853435 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316dd70e52344ea08d4b09658f48f5e0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:59.854309 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316dd70e52344ea08d4b09658f48f5e0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:59.854970 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316dd70e52344ea08d4b09658f48f5e0'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:47:59.870884 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1261/2549] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:47:59 2022] DELETE /identity/v3/users/316dd70e52344ea08d4b09658f48f5e0 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:00.047293 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:00.056562 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821db572-8385-44aa-bc9a-bcea634d7a74 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:00.113347 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:00.114003 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:00.114286 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:00.114557 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:00.120307 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:00.120474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:00.122146 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:00.206431 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:00.228184 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb8aa1-2e67-4e2b-b9f1-fa34a3d87943 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:00.229537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:00.231025 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:00.236848 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:00.237496 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:00.237819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:00.238219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] PATH_INFO: `/v3/projects/24521b04f6ec4b2ba29dca8730204b73` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:00.239388 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24521b04f6ec4b2ba29dca8730204b73)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:00.241365 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1289/2550] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:48:00.242061 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:00.303269 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24521b04f6ec4b2ba29dca8730204b73', 'request_id': 'req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.312872 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24521b04f6ec4b2ba29dca8730204b73', 'request_id': 'req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.323389 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24521b04f6ec4b2ba29dca8730204b73', 'request_id': 'req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.339714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] The token cache is being invalidate because project 24521b04f6ec4b2ba29dca8730204b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:48:00.340078 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.341025 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.345658 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1262/2551] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:48:00 2022] DELETE /identity/v3/projects/24521b04f6ec4b2ba29dca8730204b73 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:00.346992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:00.363126 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37834a-c3d1-490e-9eb1-cca483073209 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:00.423341 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:00.423975 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:00.424259 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:00.424537 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] PATH_INFO: `/v3/users/3329bbe77b3d4a878a0d2d26787429e2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:00.425710 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3329bbe77b3d4a878a0d2d26787429e2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:00.427738 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:00.478056 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3329bbe77b3d4a878a0d2d26787429e2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.485308 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3329bbe77b3d4a878a0d2d26787429e2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.499102 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3329bbe77b3d4a878a0d2d26787429e2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.508145 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3329bbe77b3d4a878a0d2d26787429e2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.509756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-82e5471a-b6a2-41e8-ace5-16a93028a34f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3329bbe77b3d4a878a0d2d26787429e2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.517075 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1290/2552] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:48:00 2022] DELETE /identity/v3/users/3329bbe77b3d4a878a0d2d26787429e2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:48:00.694522 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:00.696540 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:00.715187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0afb-b68a-4033-8c4c-c3ed26ce9430 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:00.719111 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:00.777377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:00.778047 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:00.778327 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:00.778602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] PATH_INFO: `/v3/projects/51a6bb938d7c431893f3375fe6495997` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:00.779909 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a6bb938d7c431893f3375fe6495997)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:00.781871 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:00.849050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a6bb938d7c431893f3375fe6495997', 'request_id': 'req-3a6a145e-a2e2-4607-bfd4-5f09706ec076'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.855915 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a6bb938d7c431893f3375fe6495997', 'request_id': 'req-3a6a145e-a2e2-4607-bfd4-5f09706ec076'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.866881 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a6bb938d7c431893f3375fe6495997', 'request_id': 'req-3a6a145e-a2e2-4607-bfd4-5f09706ec076'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.880724 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] The token cache is being invalidate because project 51a6bb938d7c431893f3375fe6495997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:48:00.881001 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.881644 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-3a6a145e-a2e2-4607-bfd4-5f09706ec076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:00.883219 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1263/2553] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:48:00 2022] DELETE /identity/v3/projects/51a6bb938d7c431893f3375fe6495997 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:00.888099 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee946a-1465-4196-b92b-9aba929f2909 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:00.888387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee946a-1465-4196-b92b-9aba929f2909 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:00.888662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee946a-1465-4196-b92b-9aba929f2909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:00.908558 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-16ee946a-1465-4196-b92b-9aba929f2909 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:00.909772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16ee946a-1465-4196-b92b-9aba929f2909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:00.972092 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1291/2554] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:48:00.980353 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e985c7-9768-41b6-a781-974a5e39ba6a None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:00.980645 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e985c7-9768-41b6-a781-974a5e39ba6a None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:00.980921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e985c7-9768-41b6-a781-974a5e39ba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.007647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-25e985c7-9768-41b6-a781-974a5e39ba6a None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:01.008914 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-25e985c7-9768-41b6-a781-974a5e39ba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:01.020526 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1264/2555] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:48:01.024985 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee946a-1465-4196-b92b-9aba929f2909 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.029787 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:01.036142 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee946a-1465-4196-b92b-9aba929f2909 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.044029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.044912 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.045338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.045793 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.046785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:48:01.047252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.049812 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.063256 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1292/2556] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:48:01.067494 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e985c7-9768-41b6-a781-974a5e39ba6a None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.077128 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e985c7-9768-41b6-a781-974a5e39ba6a None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.081787 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.082411 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.082691 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.082965 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.083595 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.085473 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.113961 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1265/2557] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:48:01 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:48:01.118650 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.128724 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd48832-884d-40a2-9017-9e4270808972 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.184468 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.185058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.185334 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.185603 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.186263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.188170 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.208681 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1293/2558] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:48:01 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:48:01.213513 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.223808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dc1ce-30c2-4aa5-b807-c8efcecd662f admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.229085 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.229678 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.229988 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.230260 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.230899 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.232476 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.242420 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1266/2559] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:48:01.246159 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.255954 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15514023-44ae-42e4-9351-6f56beffcbbc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.260715 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb71d61-049f-4cf3-a9c0-334d4a150d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.261294 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb71d61-049f-4cf3-a9c0-334d4a150d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.261568 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb71d61-049f-4cf3-a9c0-334d4a150d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.261871 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb71d61-049f-4cf3-a9c0-334d4a150d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.262498 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb71d61-049f-4cf3-a9c0-334d4a150d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.264081 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb71d61-049f-4cf3-a9c0-334d4a150d98 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.276579 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1294/2560] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:48:01.280499 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.290808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8c3ec-c330-440d-9cab-516db341a0b5 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.295434 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.296069 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.296349 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.296623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] PATH_INFO: `/v3/projects/7023eaea52664c67a8015ee6c12df168/users/238b45437b294addaa59cf372fa56f90/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.311544 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7023eaea52664c67a8015ee6c12df168, user_id=238b45437b294addaa59cf372fa56f90, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.314194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.327112 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1267/2561] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:48:01 2022] PUT /identity/v3/projects/7023eaea52664c67a8015ee6c12df168/users/238b45437b294addaa59cf372fa56f90/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:01.338487 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acde58-2918-4255-ad8a-0cdfc2791bd9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.338770 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acde58-2918-4255-ad8a-0cdfc2791bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.339039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acde58-2918-4255-ad8a-0cdfc2791bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.366764 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-04acde58-2918-4255-ad8a-0cdfc2791bd9 None None] MFA Rules not processed for user `238b45437b294addaa59cf372fa56f90`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:01.368599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-04acde58-2918-4255-ad8a-0cdfc2791bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:01.448509 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1295/2562] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:48:01.450453 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.489853 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:01.526714 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a07f8-89d7-42e4-aab1-81fb5968abb4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.532424 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.533209 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.533656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.534102 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.534883 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.537016 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.569446 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1268/2563] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:48:01 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:48:01.574337 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acde58-2918-4255-ad8a-0cdfc2791bd9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.587785 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acde58-2918-4255-ad8a-0cdfc2791bd9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.650208 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:01.659798 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.660553 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.660909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.661278 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.662042 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.664122 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.691626 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1296/2564] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:48:01 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:48:01.693147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.705262 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f53ade-85e4-40b8-970a-28f0964fff88 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.710375 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.711067 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.712020 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.712358 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.713071 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.714769 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.726273 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1269/2565] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:48:01.730709 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.741299 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e0cf-b02e-4b00-a56c-24e93c43fa81 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.746685 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.747397 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.747679 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.747956 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] PATH_INFO: `/v3/projects/a770f4e3710d426787c32c8990546870/users/81d83bca397b4600aaa6b7dd480e752e/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.763992 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a770f4e3710d426787c32c8990546870, user_id=81d83bca397b4600aaa6b7dd480e752e, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.766629 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.779797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1297/2566] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:48:01 2022] PUT /identity/v3/projects/a770f4e3710d426787c32c8990546870/users/81d83bca397b4600aaa6b7dd480e752e/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:48:01.783841 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.794044 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68eb41a-4f3a-4f48-9aa5-745092fab2d6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.848747 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a698c4f-3d02-4cbf-b4e2-9fff07078cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.849340 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a698c4f-3d02-4cbf-b4e2-9fff07078cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.849617 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a698c4f-3d02-4cbf-b4e2-9fff07078cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.849921 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a698c4f-3d02-4cbf-b4e2-9fff07078cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.850552 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a698c4f-3d02-4cbf-b4e2-9fff07078cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.852194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a698c4f-3d02-4cbf-b4e2-9fff07078cb7 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.861559 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1270/2567] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:01 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 14:48:01.865757 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:01.875909 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7fe31-2a56-4d36-8c36-8fd23c8f56f7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:01.880760 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:01.881399 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.881673 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.881971 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] PATH_INFO: `/v3/projects/a770f4e3710d426787c32c8990546870/users/81d83bca397b4600aaa6b7dd480e752e/roles/96af379f1477489b8542698d3c012f1a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.884380 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a770f4e3710d426787c32c8990546870, user_id=81d83bca397b4600aaa6b7dd480e752e, role_id=96af379f1477489b8542698d3c012f1a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:01.887024 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:01.898756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1298/2568] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:48:01 2022] PUT /identity/v3/projects/a770f4e3710d426787c32c8990546870/users/81d83bca397b4600aaa6b7dd480e752e/roles/96af379f1477489b8542698d3c012f1a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:01.910194 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:01.910483 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:01.910760 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:01.939655 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] MFA Rules not processed for user `81d83bca397b4600aaa6b7dd480e752e`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:01.940860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:02.010987 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1271/2569] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:48:02.023602 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:02.025439 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:02.087353 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0f6e3-1bcf-4591-a712-6c97373ef7ad admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:02.091032 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923f5de-f571-40cc-b001-20818b134033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:02.091678 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f5de-f571-40cc-b001-20818b134033 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:02.091963 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f5de-f571-40cc-b001-20818b134033 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:02.092234 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923f5de-f571-40cc-b001-20818b134033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:02.148396 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923f5de-f571-40cc-b001-20818b134033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:02.150136 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923f5de-f571-40cc-b001-20818b134033 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:02.161252 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1299/2570] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:48:04.186698 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:04.188492 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:04.221840 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:04.260385 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89d4d7-b344-42b7-a84d-1467676b325d None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:04.265187 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:04.265846 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:04.266133 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:04.266408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] PATH_INFO: `/v3/users/9c550b766a3d4ba8839e2f9e8308beb3` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:04.267556 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c550b766a3d4ba8839e2f9e8308beb3)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:04.270179 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:04.323807 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c550b766a3d4ba8839e2f9e8308beb3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.331041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c550b766a3d4ba8839e2f9e8308beb3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.343740 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c550b766a3d4ba8839e2f9e8308beb3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.344680 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c550b766a3d4ba8839e2f9e8308beb3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.345641 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c550b766a3d4ba8839e2f9e8308beb3'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.360512 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1272/2571] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:48:04 2022] DELETE /identity/v3/users/9c550b766a3d4ba8839e2f9e8308beb3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:04.365324 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:04.365838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:04.365931 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:04.390430 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:04.391508 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:04.455749 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1300/2572] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:48:04.466302 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:04.467816 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:04.533410 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742cb87-83e4-40a1-a303-4505a74ee79e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:04.537221 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c86aa-2814-411a-a66c-bd57fb887dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:04.537864 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c86aa-2814-411a-a66c-bd57fb887dde service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:04.538147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c86aa-2814-411a-a66c-bd57fb887dde service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:04.538418 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c86aa-2814-411a-a66c-bd57fb887dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:04.548976 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c86aa-2814-411a-a66c-bd57fb887dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:04.550471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c86aa-2814-411a-a66c-bd57fb887dde service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:04.560568 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1273/2573] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:48:04.705839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:04.707394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:04.725411 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:04.732479 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd6cda-eaa8-4c42-abcb-8e2c4fe691a2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:04.741662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:04.743107 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:04.743401 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:04.743680 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] PATH_INFO: `/v3/users/238b45437b294addaa59cf372fa56f90` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:04.745261 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=238b45437b294addaa59cf372fa56f90)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:04.747318 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:04.808532 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '238b45437b294addaa59cf372fa56f90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.818819 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '238b45437b294addaa59cf372fa56f90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.837039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '238b45437b294addaa59cf372fa56f90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.846756 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '238b45437b294addaa59cf372fa56f90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.847464 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '238b45437b294addaa59cf372fa56f90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:04.854593 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1301/2574] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:48:04 2022] DELETE /identity/v3/users/238b45437b294addaa59cf372fa56f90 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:04.859782 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:04.860247 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:04.860876 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:04.863828 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:04.865451 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:04.885350 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:04.888174 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:04.944016 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb1dd-72cf-417a-bd8a-834bc022c459 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:04.949418 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:04.950074 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:04.950356 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:04.950628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] PATH_INFO: `/v3/projects/dd18cba2516042da8a352a3e85faa548` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:04.952685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:04.952838 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd18cba2516042da8a352a3e85faa548)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:04.954600 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:04.973584 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1274/2575] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:48:04.983656 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:04.985089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:05.019039 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd18cba2516042da8a352a3e85faa548', 'request_id': 'req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.026271 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd18cba2516042da8a352a3e85faa548', 'request_id': 'req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.032877 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd18cba2516042da8a352a3e85faa548', 'request_id': 'req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.047836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] The token cache is being invalidate because project dd18cba2516042da8a352a3e85faa548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:48:05.048129 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.048797 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.050443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1302/2576] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:48:04 2022] DELETE /identity/v3/projects/dd18cba2516042da8a352a3e85faa548 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:05.056618 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:05.058201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:05.069972 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:05.082050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987dbc-d97f-4fe3-84ac-12239663f439 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:05.160338 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd2b1-933f-4fbd-9e79-34b778e83d90 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:05.165662 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:05.166332 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:05.166615 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:05.166886 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] PATH_INFO: `/v3/users/cd365434783a46bd90e6da9194b65ed6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:05.168047 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd365434783a46bd90e6da9194b65ed6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:05.170021 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:05.171730 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:05.172409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:05.172771 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:05.173049 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:05.174910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:05.187487 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:05.188931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:05.199687 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1275/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:48:05.219850 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd365434783a46bd90e6da9194b65ed6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.228755 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd365434783a46bd90e6da9194b65ed6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.242201 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd365434783a46bd90e6da9194b65ed6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.262839 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd365434783a46bd90e6da9194b65ed6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.263684 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd365434783a46bd90e6da9194b65ed6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.270567 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1303/2578] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:48:05 2022] DELETE /identity/v3/users/cd365434783a46bd90e6da9194b65ed6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:05.557403 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:05.559043 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:05.583860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9bdcb-10a7-4612-8c49-da51ca2f24a9 service neutron] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:05.635087 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:05.636544 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:05.656428 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:05.657081 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:05.657363 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:05.657639 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] PATH_INFO: `/v3/projects/7023eaea52664c67a8015ee6c12df168` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:05.658809 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7023eaea52664c67a8015ee6c12df168)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:05.660711 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:05.709598 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa916a-ef2a-4e8d-be21-40b76b0dbd99 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:05.709939 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7023eaea52664c67a8015ee6c12df168', 'request_id': 'req-6982e760-041a-40a9-b5f9-d8ddd6f2513a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.712050 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:05.716910 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7023eaea52664c67a8015ee6c12df168', 'request_id': 'req-6982e760-041a-40a9-b5f9-d8ddd6f2513a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.724837 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7023eaea52664c67a8015ee6c12df168', 'request_id': 'req-6982e760-041a-40a9-b5f9-d8ddd6f2513a'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.736739 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] The token cache is being invalidate because project 7023eaea52664c67a8015ee6c12df168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:48:05.737208 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.737880 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.739616 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1276/2579] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:48:05 2022] DELETE /identity/v3/projects/7023eaea52664c67a8015ee6c12df168 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:05.743855 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:05.743997 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:05.745304 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:05.773768 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:05.774492 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:05.774772 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:05.775046 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] PATH_INFO: `/v3/projects/2204bb7b6f5149568110babf8342258f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:05.776210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2204bb7b6f5149568110babf8342258f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:05.778183 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:05.811268 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982e760-041a-40a9-b5f9-d8ddd6f2513a admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:05.824428 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2204bb7b6f5149568110babf8342258f', 'request_id': 'req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.830555 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2204bb7b6f5149568110babf8342258f', 'request_id': 'req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.837232 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2204bb7b6f5149568110babf8342258f', 'request_id': 'req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.849786 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] The token cache is being invalidate because project 2204bb7b6f5149568110babf8342258f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:48:05.850377 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.850693 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.852235 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1304/2580] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:48:05 2022] DELETE /identity/v3/projects/2204bb7b6f5149568110babf8342258f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:05.856219 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:05.857628 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:05.870291 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:05.870918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:05.871200 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:05.871481 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] PATH_INFO: `/v3/users/81d83bca397b4600aaa6b7dd480e752e` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:05.872663 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d83bca397b4600aaa6b7dd480e752e)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:05.874638 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:05.922292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0d87-00f1-4ebd-bb0f-122ae0cb3db9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:05.923089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d83bca397b4600aaa6b7dd480e752e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.927257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:05.927875 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:05.928150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:05.928419 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] PATH_INFO: `/v3/users/7a3cfd5dccc24eb48a0640102ef2b60d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:05.929548 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3cfd5dccc24eb48a0640102ef2b60d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:05.931408 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d83bca397b4600aaa6b7dd480e752e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.932013 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:05.943940 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d83bca397b4600aaa6b7dd480e752e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.944733 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d83bca397b4600aaa6b7dd480e752e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.945443 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d83bca397b4600aaa6b7dd480e752e'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.962536 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1277/2581] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:48:05 2022] DELETE /identity/v3/users/81d83bca397b4600aaa6b7dd480e752e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:48:05.983730 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3cfd5dccc24eb48a0640102ef2b60d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.990809 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3cfd5dccc24eb48a0640102ef2b60d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:05.999426 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:06.005209 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3cfd5dccc24eb48a0640102ef2b60d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.014848 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3cfd5dccc24eb48a0640102ef2b60d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.015745 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3cfd5dccc24eb48a0640102ef2b60d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.022387 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1305/2582] 10.222.0.20 () {58 vars in 1224 bytes} [Tue Sep 13 14:48:05 2022] DELETE /identity/v3/users/7a3cfd5dccc24eb48a0640102ef2b60d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:48:06.311147 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.313462 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:06.332092 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:06.402795 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee15344-7c75-4100-bb37-f70ba82b7a66 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.454978 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.456591 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:06.476684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.477471 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.478192 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.478468 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] PATH_INFO: `/v3/projects/a770f4e3710d426787c32c8990546870` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.479931 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a770f4e3710d426787c32c8990546870)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.482026 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.531739 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b373ac-c97a-4b40-a473-820a2f44cb75 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.531986 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a770f4e3710d426787c32c8990546870', 'request_id': 'req-f8679c7e-b876-485e-9c3c-2dac1a54a44b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.538648 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a770f4e3710d426787c32c8990546870', 'request_id': 'req-f8679c7e-b876-485e-9c3c-2dac1a54a44b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.548178 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a770f4e3710d426787c32c8990546870', 'request_id': 'req-f8679c7e-b876-485e-9c3c-2dac1a54a44b'} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.562032 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] The token cache is being invalidate because project a770f4e3710d426787c32c8990546870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81606) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:48:06.562407 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.563118 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.notifications [None req-f8679c7e-b876-485e-9c3c-2dac1a54a44b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81606) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.564904 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1278/2583] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:48:06 2022] DELETE /identity/v3/projects/a770f4e3710d426787c32c8990546870 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:06.569323 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7895132-4696-47a6-bbe2-0a1dc9ffe806 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.569602 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7895132-4696-47a6-bbe2-0a1dc9ffe806 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.569907 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7895132-4696-47a6-bbe2-0a1dc9ffe806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.591274 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-e7895132-4696-47a6-bbe2-0a1dc9ffe806 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:06.592357 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.592495 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-e7895132-4696-47a6-bbe2-0a1dc9ffe806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:06.592983 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.593263 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.593536 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] PATH_INFO: `/v3/projects/7619df6754db47288ee1c376d5fe2202` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.594683 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=7619df6754db47288ee1c376d5fe2202)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.596587 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.600402 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1279/2584] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:48:06.608534 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3f71-9614-4e6f-8123-bba96f495776 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.608812 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3f71-9614-4e6f-8123-bba96f495776 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.609132 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3f71-9614-4e6f-8123-bba96f495776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.628788 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-bcdf3f71-9614-4e6f-8123-bba96f495776 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:06.630284 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-bcdf3f71-9614-4e6f-8123-bba96f495776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:06.638451 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1280/2585] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 14:48:06.642541 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3f71-9614-4e6f-8123-bba96f495776 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.647799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7619df6754db47288ee1c376d5fe2202', 'request_id': 'req-26879081-a75d-4ce2-8edc-8ebf553189ce'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.652729 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3f71-9614-4e6f-8123-bba96f495776 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.654220 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7619df6754db47288ee1c376d5fe2202', 'request_id': 'req-26879081-a75d-4ce2-8edc-8ebf553189ce'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.660808 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7619df6754db47288ee1c376d5fe2202', 'request_id': 'req-26879081-a75d-4ce2-8edc-8ebf553189ce'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.675033 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] The token cache is being invalidate because project 7619df6754db47288ee1c376d5fe2202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 14:48:06.675320 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.675935 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-26879081-a75d-4ce2-8edc-8ebf553189ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 14:48:06.677454 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1306/2586] 10.222.0.20 () {58 vars in 1233 bytes} [Tue Sep 13 14:48:06 2022] DELETE /identity/v3/projects/7619df6754db47288ee1c376d5fe2202 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:48:06.683230 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:06.684288 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e49e2-e21f-41f1-8c2a-1813fd0e8e34 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.684651 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e49e2-e21f-41f1-8c2a-1813fd0e8e34 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.684925 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e49e2-e21f-41f1-8c2a-1813fd0e8e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.709636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-365e49e2-e21f-41f1-8c2a-1813fd0e8e34 None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:06.710980 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-365e49e2-e21f-41f1-8c2a-1813fd0e8e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:06.719031 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.719577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.719860 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.720134 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] PATH_INFO: `/v3/domains` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.720781 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81606) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:48:06.721089 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.722719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.724705 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:06.735694 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1281/2587] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 14:48:06.741936 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.745819 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:06.763776 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b56508-562f-4925-b714-2bb9b266a462 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.768413 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.769018 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.769286 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.769710 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.770223 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.772360 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.778182 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1307/2588] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:48:06.786079 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c25e6-f7a6-44dd-b96c-eaca6b62881d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.786363 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c25e6-f7a6-44dd-b96c-eaca6b62881d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.786636 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c25e6-f7a6-44dd-b96c-eaca6b62881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.803685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1282/2589] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:48:06.804684 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.807210 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-df8c25e6-f7a6-44dd-b96c-eaca6b62881d None None] MFA Rules not processed for user `9242a7b2cdf946a38262cb2b38f5cb44`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:06.808390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-df8c25e6-f7a6-44dd-b96c-eaca6b62881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:06.867345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edf816-260e-4e19-b65c-4103c19267fe admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.869346 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1308/2590] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:48:06.872808 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.873409 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.873519 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c25e6-f7a6-44dd-b96c-eaca6b62881d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.873690 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.873994 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] PATH_INFO: `/v3/users` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.874623 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.876829 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.882258 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c25e6-f7a6-44dd-b96c-eaca6b62881d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.887088 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.887652 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.887936 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.888202 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.888836 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 14:48:06.889138 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.890855 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.899021 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1283/2591] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 14:48:06.903079 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.903460 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1309/2592] 10.222.0.20 () {58 vars in 1153 bytes} [Tue Sep 13 14:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 14:48:06.907358 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.916908 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5a473-2fd2-429c-b3c8-c1d8ce9db61e admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.917617 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69348183-c76a-4c80-a481-f892b1f39ad5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.921598 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.922215 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.922443 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.922578 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.922793 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.923010 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.923289 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.923430 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.923687 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.924335 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.925007 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.926285 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.935072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1284/2593] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:48:06.938922 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.949288 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda8081-efc2-4777-8735-d71aac090ae4 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:06.953944 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:06.957290 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1310/2594] 10.222.0.20 () {60 vars in 1153 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:48:06.957425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:06.957425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:06.957425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:06.957425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:06.957425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:06.958522 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.969158 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1285/2595] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:48:06.973240 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:06.983814 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfba32-a730-443d-be37-c36d81e1bea6 admin admin] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:07.022161 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee72adb-cd1d-4f11-b1ab-3269fe3b7783 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:07.027095 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:07.027698 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.027973 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.028244 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.028895 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:07.030854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:07.040316 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbfaf1-c00d-4c07-9a4e-94071cfe9225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:07.041175 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbfaf1-c00d-4c07-9a4e-94071cfe9225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.041555 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbfaf1-c00d-4c07-9a4e-94071cfe9225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.041929 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbfaf1-c00d-4c07-9a4e-94071cfe9225 admin admin] PATH_INFO: `/v3/projects/ae9d485b3e4f47faa611b55b618cb1c0/users/552e151a5cd44044b9a86fd887f7ae3a/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.053412 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1311/2596] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Sep 13 14:48:06 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 14:48:07.057595 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:07.058321 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbfaf1-c00d-4c07-9a4e-94071cfe9225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae9d485b3e4f47faa611b55b618cb1c0, user_id=552e151a5cd44044b9a86fd887f7ae3a, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:07.060955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbfaf1-c00d-4c07-9a4e-94071cfe9225 admin admin] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:07.063202 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:07.067830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bda12a-7aae-40d0-b6d8-292e968a927f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:07.073631 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:07.074257 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.074531 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.074799 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.075425 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:07.077002 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:07.077343 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1286/2597] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:48:06 2022] PUT /identity/v3/projects/ae9d485b3e4f47faa611b55b618cb1c0/users/552e151a5cd44044b9a86fd887f7ae3a/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 14:48:07.087533 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] REQUEST_METHOD: `POST` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.087707 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1312/2598] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:48:07.087955 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.088377 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.091470 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:07.097953 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:07.103151 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aedb5e-0744-4b1f-8f05-0c0d05044ebe admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:07.117779 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.auth.core [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] MFA Rules not processed for user `552e151a5cd44044b9a86fd887f7ae3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81606) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:07.119072 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:07.165904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:07.166632 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.167027 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.167304 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.168100 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:07.169854 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:07.181672 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1313/2599] 10.222.0.20 () {58 vars in 1122 bytes} [Tue Sep 13 14:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 14:48:07.186029 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:07.192493 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1287/2600] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:48:07.196948 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf07e4-5fdc-4395-a0e5-f374f74fd304 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:07.202916 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e77a5c-e606-4155-a736-cd02187f7ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242a7b2cdf946a38262cb2b38f5cb44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d23d643906b4c84bfbe35da532a7c31', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:07.203688 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e77a5c-e606-4155-a736-cd02187f7ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.203834 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e77a5c-e606-4155-a736-cd02187f7ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.204110 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e77a5c-e606-4155-a736-cd02187f7ca0 admin admin] PATH_INFO: `/v3/projects/5770b9d00c90412a87b9d0080e855240/users/68b14b5850594aafa28d0c5fa25fef52/roles/6bc93361bf6e4aeb91fd6c5713557177` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.205789 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:07.207474 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:07.220599 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e77a5c-e606-4155-a736-cd02187f7ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5770b9d00c90412a87b9d0080e855240, user_id=68b14b5850594aafa28d0c5fa25fef52, role_id=6bc93361bf6e4aeb91fd6c5713557177)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:07.224152 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e77a5c-e606-4155-a736-cd02187f7ca0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:07.240098 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1314/2601] 10.222.0.20 () {60 vars in 1483 bytes} [Tue Sep 13 14:48:07 2022] PUT /identity/v3/projects/5770b9d00c90412a87b9d0080e855240/users/68b14b5850594aafa28d0c5fa25fef52/roles/6bc93361bf6e4aeb91fd6c5713557177 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 14:48:07.252053 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.252394 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.252607 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.283813 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] MFA Rules not processed for user `68b14b5850594aafa28d0c5fa25fef52`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 14:48:07.285130 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:07.359582 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1315/2602] 10.222.0.20 () {58 vars in 958 bytes} [Tue Sep 13 14:48:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 14:48:07.380878 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:07.382366 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:07.421426 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537630c-be81-48d0-bff6-baea01bb5aa1 None None] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:07.443570 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f0402-1f0b-4915-926a-a29306483ee2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:07.447417 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:07.448017 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.448292 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.448559 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.457440 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:07.458940 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:07.469184 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1316/2603] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:48:07.477528 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7214e6addc646a899efb82283ededc1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:07.478418 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:07.478703 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:07.478978 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:07.513794 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:07.553098 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:07.554634 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:07.564798 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1288/2604] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 14:48:10.276511 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:10.278390 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:10.296763 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:10.403830 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33878566-fb5e-4df7-8390-40c5aeac0da2 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:10.410169 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a4ea803f6e44648bb217abd00e11c3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:10.410840 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:10.411150 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:10.411422 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:10.413424 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:10.429058 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:10.430947 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:10.443249 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1317/2605] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:48:10.509345 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:48:10.512181 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:10.531425 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0073b2-f1fd-4276-8c49-a873597140a7 service glance] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:48:10.537647 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c95553acbf23423993a4d25e83ecee48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:48:10.538283 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:48:10.538591 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:48:10.538865 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:48:10.540719 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:48:10.644197 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:48:10.645574 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:48:10.673391 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1289/2606] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:48:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:48:20.438577 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=30 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:48:20.667577 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=53 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:49:44.492789 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:49:44.494381 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:49:44.553433 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a2e1-9901-4431-b26a-79112df20113 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:49:44.557483 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cd67c-01fd-4fd4-9d61-175cc42bf8b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '11a418a2a35146b59c433421fe507500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:49:44.558116 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd67c-01fd-4fd4-9d61-175cc42bf8b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:49:44.558398 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd67c-01fd-4fd4-9d61-175cc42bf8b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:49:44.558671 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd67c-01fd-4fd4-9d61-175cc42bf8b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:49:44.560374 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a93cd67c-01fd-4fd4-9d61-175cc42bf8b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:49:44.570238 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cd67c-01fd-4fd4-9d61-175cc42bf8b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:49:44.571979 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cd67c-01fd-4fd4-9d61-175cc42bf8b3 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:49:44.579904 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1318/2607] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:49:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:49:54.576343 dv-cinder-857434-1-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=26 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 14:50:53.786990 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] Authenticating user token {{(pid=81606) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 14:50:53.788378 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:50:53.880918 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83817c68-1fa4-4588-bb6c-6413a2548694 service cinder] Validating token access rules against request {{(pid=81606) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 14:50:53.887171 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2fef7-ba64-4a9a-b8df-228428c535cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd09c0b02c24f4c5db5ed35fdee2589b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e8ba472641f4e46b3884d884882bdaf', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81606) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 14:50:53.887765 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2fef7-ba64-4a9a-b8df-228428c535cc service placement] REQUEST_METHOD: `GET` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 14:50:53.888041 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2fef7-ba64-4a9a-b8df-228428c535cc service placement] SCRIPT_NAME: `/identity` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 14:50:53.888312 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2fef7-ba64-4a9a-b8df-228428c535cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81606) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 14:50:53.890244 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.fernet_utils [None req-28a2fef7-ba64-4a9a-b8df-228428c535cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81606) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 14:50:53.903019 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2fef7-ba64-4a9a-b8df-228428c535cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 14:50:53.904565 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2fef7-ba64-4a9a-b8df-228428c535cc service placement] RBAC: Authorization granted {{(pid=81606) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 14:50:53.914245 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: [pid: 81606|app: 0|req: 1290/2608] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Sep 13 14:50:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 14:51:03.910685 dv-cinder-857434-1-iscsi devstack@keystone.service[81606]: DEBUG dbcounter [-] [81606] Writing DB stats keystone:SELECT=43 {{(pid=81606) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}